feat: consolidate docs, backend/session infra, and settings updates
This commit is contained in:
@@ -0,0 +1,169 @@
|
||||
# WP Agentic Writer Eleventh Retrace Audit
|
||||
|
||||
Audit date: 2026-05-26
|
||||
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_TENTH_PASS_2026-05-26.md`
|
||||
Scope: eleventh pass after tenth-retrace implementation, covering provider transparency coverage, model registry adoption, chat/context compatibility, cost tracking contracts, UI/UX, and release readiness.
|
||||
Status: COMPLETE / RETRACED
|
||||
Completion marker: 2026-05-26
|
||||
Follow-up retrace: `docs/architecture/PLUGIN_AUDIT_RETRACE_TWELFTH_PASS_2026-05-26.md`
|
||||
|
||||
> This eleventh-pass report has been implemented and retraced. Keep this document as historical evidence only; use the twelfth-pass report for current remaining work.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The tenth-pass implementation improved the plugin again:
|
||||
|
||||
- A shared frontend `applyProviderMetadata()` helper now exists.
|
||||
- Several AI response paths call that helper, so provider/fallback metadata reaches the UI in more than just the original stream completion path.
|
||||
- The P0 failed-attempt fatal from the ninth pass remains fixed.
|
||||
- PHP and JavaScript syntax checks pass.
|
||||
|
||||
No new P0 blocker was found. The remaining issues are now narrower, but still real:
|
||||
|
||||
- Provider metadata UI coverage is broader, but still not complete across all AI response paths.
|
||||
- Model registry adoption is improved, but some active and fallback paths still carry hard-coded model IDs.
|
||||
- The sidebar still hydrates chat through deprecated `/chat-history`.
|
||||
- Raw cost hook calls still bypass the `track_ai_cost()` helper in many places.
|
||||
- Live WordPress editor browser verification is still pending.
|
||||
|
||||
## Verification Performed
|
||||
|
||||
- PHP syntax check across plugin PHP files: passed.
|
||||
- `node -c assets/js/sidebar.js`: passed.
|
||||
- `node -c assets/js/settings-v2.js`: passed.
|
||||
- `node -c assets/js/sidebar-utils.js`: passed.
|
||||
- Static retrace of tenth-pass findings against current code.
|
||||
- Static sweep of provider metadata UI usage, model registry adoption, chat-history usage, and raw cost hooks.
|
||||
- No live WordPress editor/browser workflow was run in this pass.
|
||||
|
||||
## Tenth-Pass Status Trace
|
||||
|
||||
| Tenth-pass item | Current status | Evidence |
|
||||
|---|---:|---|
|
||||
| Shared frontend provider metadata helper | Fixed | `applyProviderMetadata()` exists at `assets/js/sidebar.js:75-91`. |
|
||||
| Provider metadata UI coverage | Improved, still partial | Helper is called at `assets/js/sidebar.js:1043`, `1772`, `1995`, `2281`, and `3349`, but some AI response paths still do not call it. |
|
||||
| Provider badge rendering | Fixed for covered paths | Provider/fallback badge renders near cost at `assets/js/sidebar.js:4677-4720`. |
|
||||
| Model registry adoption | Improved, still partial | Active settings/sidebar paths use `WPAW_Model_Registry`, but fallback model lists, JS presets, provider property defaults, and some image paths still hard-code model IDs. |
|
||||
| Sidebar `/chat-history` dependency | Still open | `assets/js/sidebar.js:666-680` still fetches `/chat-history/${postId}`. |
|
||||
| `/chat-history` docblock mismatch | Still open | Docblock still says the endpoint does not use the conversations table at `includes/class-gutenberg-sidebar.php:1337-1339`. |
|
||||
| Raw cost hook drift | Still open | Direct `do_action( 'wp_aw_after_api_request', ... )` calls remain in `includes/class-gutenberg-sidebar.php` outside `track_ai_cost()`. |
|
||||
| Browser verification | Still open | Syntax checks passed, but no live editor workflow was verified. |
|
||||
|
||||
## Remaining Findings
|
||||
|
||||
### P1: Provider Metadata UI Coverage Is Still Partial
|
||||
|
||||
The new frontend helper is good:
|
||||
|
||||
- `applyProviderMetadata()` supports both `provider_metadata` and top-level provider fields at `assets/js/sidebar.js:75-91`.
|
||||
- It is called from several important paths, including streaming completion and some JSON responses at `assets/js/sidebar.js:1043`, `1772`, `1995`, `2281`, and `3349`.
|
||||
- The provider/fallback badge is rendered at `assets/js/sidebar.js:4677-4720`.
|
||||
|
||||
However, not every AI response path applies provider metadata yet. Examples:
|
||||
|
||||
- Meta generation parses JSON at `assets/js/sidebar.js:595-605` but does not call `applyProviderMetadata()`.
|
||||
- Summarize context parses JSON at `assets/js/sidebar.js:1601-1612` but does not call it.
|
||||
- Intent detection parses JSON at `assets/js/sidebar.js:1644-1648` but does not call it.
|
||||
- Reformat blocks parses JSON at `assets/js/sidebar.js:2191-2219` but does not call it.
|
||||
- Refine-from-chat streaming parses data events at `assets/js/sidebar.js:2771-2828` but does not apply metadata on completion.
|
||||
|
||||
Impact:
|
||||
|
||||
- The provider badge can be stale after some AI actions.
|
||||
- Users may see provider information for generation/chat but not for meta, summarization, intent, reformat, or refinement workflows.
|
||||
- This still falls short of a consistent provider transparency contract.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Call `applyProviderMetadata(data)` immediately after every AI JSON response parse.
|
||||
- Call it on every streaming `complete` event, including refine-from-chat.
|
||||
- Add a quick static check that every fetch to an AI endpoint either calls `applyProviderMetadata()` or explicitly comments why provider metadata is not expected.
|
||||
|
||||
### P1: Model Registry Still Is Not The Sole Source Of Truth
|
||||
|
||||
Registry adoption improved in active settings and sidebar defaults, but hard-coded model IDs remain in several places:
|
||||
|
||||
- `includes/class-settings-v2.php:188-215` still contains fallback model arrays with literal model IDs.
|
||||
- `includes/class-settings-v2.php:224-230` still uses literal fallback IDs in model transformation.
|
||||
- `assets/js/settings-v2.js:32-58` still hard-codes budget/balanced/premium preset IDs.
|
||||
- `includes/class-openrouter-provider.php:29-75` still hard-codes provider property defaults, and the constructor uses those properties when settings are absent at `includes/class-openrouter-provider.php:437-448`.
|
||||
- `includes/class-image-manager.php:409-478` still hard-codes image model fallbacks.
|
||||
- Legacy `includes/class-settings.php` still contains hard-coded defaults and may be instantiated if Settings V2 is unavailable.
|
||||
|
||||
Impact:
|
||||
|
||||
- The registry can still drift from runtime behavior.
|
||||
- The settings UI fallback list can disagree with generation defaults.
|
||||
- Future model changes still require touching multiple locations.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Replace remaining active runtime fallbacks with `WPAW_Model_Registry::get_default_model()` or `get_fallback_model()`.
|
||||
- Treat JS presets as curated presets and document them as such, or generate them from localized registry data.
|
||||
- Initialize OpenRouter provider defaults from the registry in the constructor.
|
||||
- Replace image manager fallback literals at `includes/class-image-manager.php:409-478`.
|
||||
- Decide whether legacy `class-settings.php` is supported; if yes, update its defaults to use the registry, otherwise remove fallback instantiation.
|
||||
|
||||
### P2: Sidebar Still Uses Deprecated `/chat-history`
|
||||
|
||||
The route compatibility bug from earlier passes appears fixed, but the sidebar still uses the deprecated route:
|
||||
|
||||
- `assets/js/sidebar.js:666-680` fetches `/chat-history/${postId}`.
|
||||
- The backend route remains registered at `includes/class-gutenberg-sidebar.php:346-354`.
|
||||
- The docblock still says the endpoint does not use conversations at `includes/class-gutenberg-sidebar.php:1337-1339`, even though the implementation now reads session-backed history.
|
||||
|
||||
Impact:
|
||||
|
||||
- The UI still depends on a compatibility endpoint.
|
||||
- Documentation and behavior disagree.
|
||||
- Future cleanup can break chat hydration again.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Move sidebar hydration to the canonical conversation/session context endpoint.
|
||||
- If `/chat-history` remains, update the docblock and response contract to explicitly say it returns session-backed compatibility data.
|
||||
|
||||
### P2: Cost Tracking Helper Is Not Yet Enforced
|
||||
|
||||
`track_ai_cost()` exists, but raw cost hook calls remain:
|
||||
|
||||
- `includes/class-gutenberg-sidebar.php` still has many direct `do_action( 'wp_aw_after_api_request', ... )` calls outside the helper.
|
||||
- `includes/class-keyword-suggester.php:122` also calls the cost hook directly.
|
||||
|
||||
Impact:
|
||||
|
||||
- New changes can still bypass provider/session/status normalization.
|
||||
- The codebase still relies on manual discipline instead of enforcing the cost tracking contract.
|
||||
|
||||
Recommended fix:
|
||||
|
||||
- Convert remaining route-level raw hooks to `track_ai_cost()`.
|
||||
- Either expose a shared cost helper outside the sidebar class or document why non-sidebar callers may use the raw hook.
|
||||
- Add a static guard that only allows raw `wp_aw_after_api_request` calls in approved files/lines.
|
||||
|
||||
### P2: Live Editor Browser Verification Still Remains
|
||||
|
||||
No live WordPress editor browser workflow was run in this retrace.
|
||||
|
||||
Recommended browser checklist:
|
||||
|
||||
- Sidebar opens and persists in the block editor.
|
||||
- Chat session continues after page reload.
|
||||
- Provider/fallback warnings render after every AI action with metadata.
|
||||
- Cost display updates after chat, plan, refine, and meta actions.
|
||||
- Unauthorized post access fails cleanly.
|
||||
- Model settings changes reflect in generated requests.
|
||||
|
||||
## Recommended Next Work
|
||||
|
||||
1. Add `applyProviderMetadata()` to the remaining AI response paths.
|
||||
2. Finish model registry adoption in active runtime paths or explicitly document curated exceptions.
|
||||
3. Move sidebar chat hydration off `/chat-history`, or update the route contract/docblock.
|
||||
4. Convert remaining raw cost hooks or add a static guard for approved direct hook use.
|
||||
5. Run the live WordPress editor browser workflow pass.
|
||||
|
||||
## Current Verdict
|
||||
|
||||
The tenth-pass implementation is proper for the provider-UI helper it targeted and does not introduce a new P0. The audit chain is now mostly down to consistency, cleanup, and browser validation.
|
||||
|
||||
I would not call the plugin fully audit-clean yet, but the remaining issues are bounded and should be much smaller to close than the earlier authorization/context/cost blockers.
|
||||
Reference in New Issue
Block a user