feat: consolidate docs, backend/session infra, and settings updates
This commit is contained in:
175
docs/architecture/PLUGIN_AUDIT_RETRACE_TENTH_PASS_2026-05-26.md
Normal file
175
docs/architecture/PLUGIN_AUDIT_RETRACE_TENTH_PASS_2026-05-26.md
Normal file
@@ -0,0 +1,175 @@
|
||||
# WP Agentic Writer Tenth Retrace Audit
|
||||
|
||||
Status: COMPLETE / RETRACED
|
||||
Completion marker: 2026-05-26
|
||||
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_ELEVENTH_PASS_2026-05-26.md`
|
||||
|
||||
Audit date: 2026-05-26
|
||||
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_NINTH_PASS_2026-05-26.md`
|
||||
Scope: tenth pass after ninth-retrace implementation, covering failed-attempt tracking, provider transparency, model registry adoption, chat/context compatibility, cost tracking contracts, UI/UX, and release readiness.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The ninth-pass implementation fixed the P0 runtime blocker:
|
||||
|
||||
- The failed-attempt branches no longer call `$provider_result->get_default_model()`.
|
||||
- They now use `WPAW_Model_Registry::get_default_model()` for writing, clarity, and refinement paths.
|
||||
- PHP and JavaScript syntax checks pass.
|
||||
|
||||
The implementation also improved two other areas:
|
||||
|
||||
- Provider metadata now reaches the sidebar in the streaming completion path and is rendered as a compact provider/fallback badge near cost.
|
||||
- Model registry adoption improved across active settings paths, sidebar defaults, activation defaults, image-manager analysis/prompt paths, and failed-attempt model fallbacks.
|
||||
|
||||
No new P0 blocker was found in this retrace. The remaining gaps are now mostly completion and verification work:
|
||||
|
||||
- Provider metadata UI is only wired for the streaming completion path, not every non-streaming AI response path.
|
||||
- The model registry exists and is partially adopted, but some model defaults/fallbacks are still hard-coded.
|
||||
- The sidebar still hydrates chat from the deprecated `/chat-history` compatibility route.
|
||||
- Raw cost hook calls still exist outside `track_ai_cost()`.
|
||||
- Live WordPress editor browser verification is still pending.
|
||||
|
||||
## Verification Performed
|
||||
|
||||
- PHP syntax check across plugin PHP files: passed.
|
||||
- `node -c assets/js/sidebar.js`: passed.
|
||||
- `node -c assets/js/settings-v2.js`: passed.
|
||||
- `node -c assets/js/sidebar-utils.js`: passed.
|
||||
- Static retrace of ninth-pass findings against current code.
|
||||
- Static sweep of failed-attempt tracking, provider metadata UI usage, model registry adoption, chat-history usage, and raw cost hooks.
|
||||
- No live WordPress editor/browser workflow was run in this pass.
|
||||
|
||||
## Ninth-Pass Status Trace
|
||||
|
||||
| Ninth-pass item | Current status | Evidence |
|
||||
|---|---:|---|
|
||||
| P0 failed-attempt fatal from `$provider_result->get_default_model()` | Fixed | Failed branches now call `WPAW_Model_Registry::get_default_model()` at `includes/class-gutenberg-sidebar.php:3796-3805`, `4163-4172`, `7004-7012`, and `7102-7110`. |
|
||||
| Failed-attempt cost helper | Improved | `track_ai_cost()` exists at `includes/class-gutenberg-sidebar.php:966-1005` and is used in sampled failure branches. |
|
||||
| Provider metadata backend envelope | Improved | Chat adds `provider_metadata` at `includes/class-gutenberg-sidebar.php:1114`, while many backend responses already include the same envelope. |
|
||||
| Provider metadata editor rendering | Partially fixed | Sidebar captures provider metadata from streaming `complete` events at `assets/js/sidebar.js:1023-1031` and renders a provider/fallback badge at `assets/js/sidebar.js:4658-4665` and `assets/js/sidebar.js:4701-4708`. |
|
||||
| Model registry adoption | Improved but incomplete | Active settings defaults use `WPAW_Model_Registry` in `includes/class-settings-v2.php:100-111`, `989-994`, and `1109-1114`; hard-coded defaults remain elsewhere. |
|
||||
| Sidebar dependency on `/chat-history` | Still open | `assets/js/sidebar.js:644-668` still fetches `/chat-history/${postId}`. |
|
||||
| Raw cost hook drift | Still open | Many direct `do_action( 'wp_aw_after_api_request', ... )` calls remain in `includes/class-gutenberg-sidebar.php`. |
|
||||
| WordPress editor browser pass | Still open | Syntax checks passed, but no live editor workflow was verified. |
|
||||
|
||||
## Remaining Findings
|
||||
|
||||
### P1: Provider Metadata UI Is Only Partially Wired
|
||||
|
||||
The sidebar now has provider metadata state and renders it:
|
||||
|
||||
- `providerInfo` state exists at `assets/js/sidebar.js:73`.
|
||||
- Streaming completion events capture `data.provider` or `data.provider_metadata` at `assets/js/sidebar.js:1023-1031`.
|
||||
- A provider/fallback badge renders near cost in the focus keyword UI at `assets/js/sidebar.js:4658-4665` and `assets/js/sidebar.js:4701-4708`.
|
||||
|
||||
The gap is coverage. The only `setProviderInfo()` call found is in the streaming completion handler. Many non-streaming fetch flows parse JSON responses that may include `provider_metadata`, but they do not appear to update `providerInfo`:
|
||||
|
||||
- Plan generation/revision non-stream paths.
|
||||
- Chat non-stream responses if enabled.
|
||||
- Meta generation.
|
||||
- Summarize context and intent detection.
|
||||
- Refine/reformat utility calls.
|
||||
|
||||
Impact:
|
||||
|
||||
- Provider transparency is visible only for some workflows.
|
||||
- Users can still run AI actions with provider metadata in the response but no visible provider/fallback update in the sidebar.
|
||||
- The Definition of Done expects actual provider visibility across AI actions, not only one streaming flow.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Add one frontend helper, for example `applyProviderMetadata(responseData)`, and call it after every AI JSON response and stream completion.
|
||||
- Support both `provider_metadata` and top-level provider fields while backend response shapes continue to coexist.
|
||||
- Add a small UI checklist or browser assertion that provider info updates after chat, plan, refine, meta, and utility actions.
|
||||
|
||||
### P1: Model Registry Adoption Is Improved But Still Not A Single Source Of Truth
|
||||
|
||||
The registry is now more than a stub:
|
||||
|
||||
- `WPAW_Model_Registry` exists in `includes/class-model-registry.php`.
|
||||
- Activation uses registry defaults at `wp-agentic-writer.php:140-145`.
|
||||
- Active settings V2 localization and sanitization use registry defaults at `includes/class-settings-v2.php:100-111` and `includes/class-settings-v2.php:989-994`.
|
||||
- Sidebar defaults use registry defaults at `includes/class-gutenberg-sidebar.php:278-283`.
|
||||
- Image-manager analysis/prompt paths now use registry defaults at `includes/class-image-manager.php:183-249`.
|
||||
|
||||
But several hard-coded model defaults or model lists remain:
|
||||
|
||||
- `includes/class-settings-v2.php:188-215` still has fallback model arrays with literal model ids.
|
||||
- `includes/class-settings-v2.php:224-230` still uses literal fallback ids in model transformation.
|
||||
- `assets/js/settings-v2.js:32-58` still hard-codes budget/balanced/premium preset ids.
|
||||
- `includes/class-openrouter-provider.php:29-75` still hard-codes provider property defaults, even though comments say they are registry-sourced.
|
||||
- `includes/class-image-manager.php:409-478` still hard-codes image-model fallback values.
|
||||
- Legacy `includes/class-settings.php` still contains old hard-coded defaults, and `wp-agentic-writer.php:101-104` can still instantiate it if Settings V2 is unavailable.
|
||||
|
||||
Impact:
|
||||
|
||||
- Runtime defaults can still drift from registry defaults.
|
||||
- Fallback UI/model lists can disagree with actual generation defaults.
|
||||
- The registry is useful now, but it is not yet enforcing the "single source of truth" claim.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Replace remaining fallback literals in active runtime paths with `WPAW_Model_Registry::get_default_model()` or `get_fallback_model()`.
|
||||
- Decide whether JS presets are curated product presets or registry-derived defaults. If curated, document them outside the "single source of truth" claim.
|
||||
- Initialize OpenRouter default properties from the registry in the constructor, or remove property defaults as authoritative values.
|
||||
- Either update legacy `class-settings.php` or clearly mark it as inactive/deprecated and remove fallback instantiation.
|
||||
- Add a static check that flags task-default strings outside the registry, except approved curated presets and pricing maps.
|
||||
|
||||
### P2: Sidebar Still Depends On Deprecated `/chat-history`
|
||||
|
||||
The earlier data-loss bug is fixed, but the frontend still hydrates chat through a deprecated compatibility endpoint:
|
||||
|
||||
- `assets/js/sidebar.js:644-668` calls `/chat-history/${postId}`.
|
||||
- The backend compatibility route remains registered at `includes/class-gutenberg-sidebar.php:346-354`.
|
||||
|
||||
Impact:
|
||||
|
||||
- The UI still depends on a route whose name and docblock communicate legacy post-meta history.
|
||||
- Future cleanup could accidentally break chat hydration.
|
||||
- It keeps one old mental model alive even though conversations are now session-backed.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Move sidebar hydration to the canonical conversation/session context endpoint.
|
||||
- If `/chat-history` remains, update its docblock and response contract to make clear that it returns session-backed compatibility data.
|
||||
|
||||
### P2: Raw Cost Hook Calls Still Bypass The New Helper
|
||||
|
||||
The new `track_ai_cost()` helper is a good step, but direct `do_action( 'wp_aw_after_api_request', ... )` calls remain throughout `includes/class-gutenberg-sidebar.php`.
|
||||
|
||||
Impact:
|
||||
|
||||
- New work can still reintroduce incomplete provider/session/status tracking.
|
||||
- Cost tracking consistency still depends on manual discipline.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Convert direct cost hook calls in `class-gutenberg-sidebar.php` to `track_ai_cost()`.
|
||||
- Add a static guard that only allows raw `wp_aw_after_api_request` in `track_ai_cost()` and the cost tracker registration.
|
||||
|
||||
### P2: Live Editor Browser Verification Still Remains
|
||||
|
||||
The screenshot correctly identifies browser verification as the remaining manual task. Static checks cannot prove the editor workflow works.
|
||||
|
||||
Recommended browser checklist:
|
||||
|
||||
- Sidebar opens and persists in the block editor.
|
||||
- Chat session continues after page reload.
|
||||
- Provider/fallback warnings render when metadata exists.
|
||||
- Cost display updates after chat, plan, refine, and meta actions.
|
||||
- Unauthorized post access fails cleanly.
|
||||
- Model settings changes reflect in generated requests.
|
||||
|
||||
## Recommended Next Work
|
||||
|
||||
1. Add a shared frontend `applyProviderMetadata()` helper and call it for every AI response path.
|
||||
2. Finish model registry adoption in active runtime paths and document any intentionally curated model presets.
|
||||
3. Move sidebar chat hydration off `/chat-history`, or update that compatibility endpoint's contract and docblock.
|
||||
4. Convert remaining raw cost hooks to `track_ai_cost()` and add a static guard.
|
||||
5. Run the live WordPress editor browser workflow pass.
|
||||
|
||||
## Current Verdict
|
||||
|
||||
The ninth-pass implementation is substantially better and fixes the P0 runtime fatal. I would mark the ninth-pass blocker complete.
|
||||
|
||||
The plugin is close to leaving the audit-chain loop, but not fully closed. The remaining work is mostly consistency and verification: complete provider metadata UI coverage, finish model registry adoption, remove or formalize the deprecated chat-history dependency, and run the live editor pass.
|
||||
Reference in New Issue
Block a user