dwindown/wp-agentic-writer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: consolidate docs, backend/session infra, and settings updates

Browse Source
This commit is contained in:
Dwindi Ramadhana
2026-05-28 00:58:20 +07:00
parent 2424acf726
commit 44e06eed88
102 changed files with 35423 additions and 11181 deletions
Download Patch File Download Diff File Expand all files Collapse all files

791
docs/architecture/AGENTIC_AUDIT_REPORT.md Normal file
View File

@@ -0,0 +1,791 @@
# WP Agentic Writer - Comprehensive Agentic Audit Report
**Audit Date:** January 21, 2026
**Auditor:** Cascade AI
**Plugin Version:** 0.1.0
**Goal:** Evaluate "Agentic-like IDE" capabilities in WordPress Gutenberg editor
---
## Executive Summary
WP Agentic Writer is a sophisticated AI-powered writing assistant with a **solid foundation** for agentic workflows. The plugin successfully implements the core "plan-first" approach: `Scribble → Research → Plan → Execute → Refine`. However, several gaps exist between the current implementation and a truly **IDE-like agentic experience**.
**Overall Agentic Score: 7.5/10**
### Strengths
- ✅ Multi-phase workflow (planning → execution)
- ✅ Clarification quiz for context gathering
- ✅ `@mention` system for block targeting (IDE-like)
- ✅ Slash commands (`/add below`, `/add above`, `/append code`)
- ✅ Real-time streaming with timeline progress
- ✅ Section-aware refinement with context
- ✅ Plan preview with diff-style actions
- ✅ Cost tracking and budget management
### Gaps Identified
- ⚠️ No undo/redo for AI changes
- ⚠️ No diff view before applying changes
- ⚠️ No "Accept/Reject" workflow for refinements
- ⚠️ Limited autonomous decision-making
- ⚠️ No agent memory across sessions (post-level only)
- ⚠️ No multi-step autonomous execution
- ⚠️ Missing keyboard shortcuts for power users
---
## Part 1: Current Architecture Analysis
### 1.1 Core Workflow Trace
```
┌─────────────────────────────────────────────────────────────────┐
│ USER INPUT (Chat Sidebar) │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ CLARITY CHECK (if enabled) │
│ - Evaluates 7 context categories │
│ - Generates clarification quiz if confidence < threshold │
│ - Categories: outcome, audience, tone, depth, expertise, │
│ content type, POV │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ MODE DETECTION │
│ - "writing" mode → Full article generation │
│ - "planning" mode → Outline only │
│ - Refinement detected → Chat refinement flow │
│ - Insert command detected → Add block flow │
└─────────────────────────────────────────────────────────────────┘
│
┌───────────────┴───────────────┐
▼ ▼
┌──────────────────────┐ ┌──────────────────────┐
│ PLAN GENERATION │ │ CHAT REFINEMENT │
│ - Stream outline │ │ - @mention resolve │
│ - Section breakdown │ │ - Edit plan create │
│ - Auto-execute │ │ - Block replacement │
└──────────────────────┘ └──────────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────────┐
│ GUTENBERG BLOCK OPERATIONS │
│ - createBlock(), insertBlocks(), replaceBlocks() │
│ - Section tracking (sectionBlocksRef) │
│ - Real-time editor updates │
└─────────────────────────────────────────────────────────────────┘
```
### 1.2 File Structure Analysis
| File | Purpose | Lines | Agentic Features |
|------|---------|-------|------------------|
| `sidebar.js` | Main UI + logic | 4,359 | @mentions, slash commands, plan execution |
| `class-gutenberg-sidebar.php` | REST API + AI calls | 4,274 | Streaming, refinement, memory |
| `class-openrouter-provider.php` | AI provider | 566 | Multi-model, web search |
| `block-refine.js` | Toolbar integration | 115 | @chat button on blocks |
| `sidebar.css` | Styling | 1,817 | Timeline, quiz UI |
### 1.3 Agent Modes
| Mode | Description | Agentic Level |
|------|-------------|---------------|
| **Writing** | Full article generation from prompt | ⭐⭐⭐ Medium |
| **Planning** | Outline-only with manual execution | ⭐⭐ Low |
| **Refinement** | Block-level changes via @mentions | ⭐⭐⭐⭐ High |
---
## Part 2: UI/UX Analysis
### 2.1 Sidebar Interface
**Current Structure:**
```
┌─────────────────────────────────────────┐
│ 💬 Chat │ ⚙️ Config │ 💰 Cost │ ← Tab Navigation
├─────────────────────────────────────────┤
│ [Status Bar - Mode + Cost] │
├─────────────────────────────────────────┤
│ ┌─────────────────────────────────────┐ │
│ │ Messages Area (scrollable) │ │
│ │ - User messages │ │
│ │ - Assistant responses │ │
│ │ - Timeline entries (progress) │ │
│ │ - Plan cards │ │
│ │ - Error messages │ │
│ └─────────────────────────────────────┘ │
├─────────────────────────────────────────┤
│ ┌─────────────────────────────────────┐ │
│ │ Input Area │ │
│ │ - Mode selector │ │
│ │ - Textarea with @mention support │ │
│ │ - Send button │ │
│ └─────────────────────────────────────┘ │
└─────────────────────────────────────────┘
```
**✅ Good:**
- Dark theme matches IDE aesthetic
- Monospace fonts for code-like feel
- Timeline progress shows "agent thinking"
- @mention autocomplete is discoverable
**❌ Issues:**
1. **No keyboard shortcuts** - Power users expect Cmd+Enter to send
2. **No command palette** - IDEs have Cmd+Shift+P for quick actions
3. **Config tab is underutilized** - Only article length selector
4. **No block outline view** - Can't see article structure at a glance
### 2.2 Block Toolbar Integration
**Current:** `@chat` button in block toolbar sends mention to sidebar
**Issue:** The button label is just "@chat" - unclear what it does
**Recommendation:** Rename to "Refine with AI" or add tooltip
### 2.3 Clarification Quiz UX
**Current Flow:**
1. Quiz appears as modal overlay in chat
2. Radio buttons for predefined options
3. Progress bar shows completion
4. Skip button available
**✅ Good:**
- Predefined options reduce friction
- Progress indicator is clear
- Fallback questions when AI fails
**❌ Issues:**
1. **No "Don't ask again" option** - Users may want to skip permanently
2. **Quiz interrupts flow** - Could be inline instead of modal
3. **No learning from previous posts** - Always starts fresh
---
## Part 3: Functionality Deep Dive
### 3.1 Generation Flow
**Strengths:**
- Streaming responses with real-time timeline
- Section-by-section execution
- Automatic title update from AI
- Resume capability after errors
**Gaps:**
1. **No preview before insertion** - Blocks appear directly
2. **No staged commits** - Can't review all changes before applying
3. **No branch/version history** - Can't revert to previous state
### 3.2 Refinement System
**Supported Commands:**
| Command | Action |
|---------|--------|
| `@this` | Current selected block |
| `@previous` | Block before current |
| `@next` | Block after current |
| `@all` | All content blocks |
| `@paragraph-N` | Nth paragraph |
| `@heading-N` | Nth heading |
| `@list-N` | Nth list |
| `/add below @block` | Insert paragraph below |
| `/add above @block` | Insert paragraph above |
| `/append code @block` | Insert code block |
| `/reformat @block` | Convert markdown to blocks |
**✅ This is very IDE-like!**
**Gaps:**
1. **No `@code-N`** - Can't target code blocks directly
2. **No `@image-N`** - Can't target images
3. **No range selection** - Can't say `@paragraph-1:3` for range
4. **No diff preview** - Changes apply immediately
### 3.3 Edit Plan System
**Current:**
```javascript
// Edit plan structure
{
"summary": "short summary",
"actions": [
{"action": "keep", "blockId": "..."},
{"action": "replace", "blockId": "...", "blockType": "...", "content": "..."},
{"action": "insert_after", "blockId": "...", "blockType": "...", "content": "..."},
{"action": "delete", "blockId": "..."}
]
}
```
**✅ Good:**
- Diff-style action preview
- Click-to-scroll to target block
- Execute/Cancel buttons
**❌ Issues:**
1. **All-or-nothing execution** - Can't apply individual actions
2. **No partial accept** - Can't accept some, reject others
3. **No inline editing** - Can't modify plan before applying
### 3.4 Memory System
**Current:**
- Post-level memory stored in `_wpaw_memory` meta
- Contains: summary, last_prompt, last_intent
- Chat history persisted per post
**Gaps:**
1. **No global memory** - Can't learn user preferences across posts
2. **No style guide storage** - Can't save writing style preferences
3. **No context from other posts** - Can't reference previous work
---
## Part 4: Agentic Gaps & Recommendations
### 4.1 Critical Missing Features (High Priority)
#### 4.1.1 Undo/Redo for AI Changes
**Problem:** No way to revert AI changes without manual Cmd+Z
**Solution:**
```javascript
// Add undo stack for AI operations
const aiUndoStack = [];
const executeWithUndo = (operation) => {
const snapshot = captureEditorState();
aiUndoStack.push(snapshot);
operation();
};
// UI: Add "Undo AI" button in timeline entries
```
#### 4.1.2 Diff View Before Apply
**Problem:** Users can't see what will change before it happens
**Solution:**
- Add "Preview Changes" mode
- Show side-by-side or inline diff
- GitHub-style green/red highlighting
#### 4.1.3 Accept/Reject Workflow
**Problem:** Changes apply immediately with no approval
**Solution:**
```
┌─────────────────────────────────────────┐
│ AI suggests: Replace paragraph 3 │
│ │
│ Before: "The old content..." │
│ After: "The new content..." │
│ │
│ [Accept] [Reject] [Edit] [Skip] │
└─────────────────────────────────────────┘
```
### 4.2 Agentic Enhancements (Medium Priority)
#### 4.2.1 Autonomous Multi-Step Execution
**Current:** User must approve each step
**Agentic:** Agent completes entire task autonomously
**Recommendation:** Add "Full Auto" mode
```javascript
const agentModes = {
'supervised': 'Approve each change', // Current
'semi-auto': 'Approve plan, auto-execute', // New
'full-auto': 'Complete task autonomously' // New (advanced)
};
```
#### 4.2.2 Agent Memory & Learning
**Current:** No learning across sessions
**Agentic:** Remember user preferences, writing style
**Recommendation:**
```php
// Global user preferences in wp_usermeta
update_user_meta($user_id, '_wpaw_preferences', [
'tone' => 'professional',
'avoid_words' => ['leverage', 'synergy'],
'preferred_length' => 'medium',
'always_include' => ['code examples'],
]);
```
#### 4.2.3 Context-Aware Suggestions
**Current:** Agent only responds to commands
**Agentic:** Agent proactively suggests improvements
**Recommendation:**
- Analyze article on idle
- Suggest improvements in sidebar
- "I noticed paragraph 3 could be clearer. Want me to refine it?"
### 4.3 IDE-Like Features (Medium Priority)
#### 4.3.1 Keyboard Shortcuts
| Shortcut | Action |
|----------|--------|
| `Cmd+Enter` | Send message |
| `Cmd+Shift+P` | Command palette |
| `Cmd+/` | Quick refine selected block |
| `Cmd+G` | Generate from selection |
| `Escape` | Cancel current operation |
#### 4.3.2 Command Palette
```
┌─────────────────────────────────────────┐
│ > _ │
├─────────────────────────────────────────┤
│ 📝 Generate article from prompt │
│ ✏️ Refine selected block │
│ 📋 Create outline only │
│ 🔄 Regenerate current section │
│ 🌐 Enable web search │
│ ⚙️ Open settings │
└─────────────────────────────────────────┘
```
#### 4.3.3 Block Outline Panel
```
┌─────────────────────────────────────────┐
│ ARTICLE STRUCTURE │
├─────────────────────────────────────────┤
│ ▼ Introduction │
│ ├─ paragraph-1 │
│ └─ paragraph-2 │
│ ▼ Getting Started │
│ ├─ heading-2 │
│ ├─ paragraph-3 │
│ └─ code-1 │
│ ▼ Advanced Usage │
│ ├─ heading-3 │
│ └─ list-1 │
└─────────────────────────────────────────┘
```
### 4.4 UX Improvements (Lower Priority)
#### 4.4.1 Inline Refinement
**Current:** Must use sidebar for all refinements
**Improvement:** Click block → inline popover with quick actions
#### 4.4.2 Streaming Preview
**Current:** Content appears in editor directly
**Improvement:** Show in preview pane first, then "Apply All"
#### 4.4.3 Smart Suggestions Bar
Show contextual actions based on selection:
```
┌─────────────────────────────────────────┐
│ 💡 Make concise │ 🔄 Rephrase │ 📝 Expand │
└─────────────────────────────────────────┘
```
---
## Part 5: Technical Debt & Code Quality
### 5.1 Identified Issues
1. **`sidebar.js` is 4,359 lines** - Should be split into modules
2. **Mixed concerns** - UI, state, API calls in same file
3. **No TypeScript** - Type safety would prevent bugs
4. **Hardcoded strings** - Should use i18n throughout
### 5.2 Recommendations
1. **Modularize sidebar.js:**
- `hooks/useChat.js`
- `hooks/usePlan.js`
- `hooks/useRefinement.js`
- `components/ChatTab.js`
- `components/ConfigTab.js`
- `utils/blockHelpers.js`
2. **Add error boundaries** - React error boundaries for graceful failures
3. **Implement proper state management** - Consider Redux or Zustand
---
## Part 6: Prioritized Action Items
### Tier 1: Critical (Do First)
| Item | Effort | Impact | Status |
|------|--------|--------|--------|
| Add Undo for AI changes | Medium | High | ❌ Not Implemented |
| Add Accept/Reject workflow | Medium | High | ✅ Implemented (Apply/Cancel buttons) |
| Add Cmd+Enter to send | Low | Medium | ✅ Implemented (line 2326 sidebar.js) |
| Add diff preview for edit plans | Medium | High | ✅ Implemented (edit_plan type with before/after) |
> **Note:** Cross-verified on Jan 21, 2026. Only **Undo for AI changes** remains to be implemented in Tier 1.
### Tier 2: Important (Do Next)
| Item | Effort | Impact |
|------|--------|--------|
| Command palette (Cmd+Shift+P) | Medium | High |
| Per-action accept/reject in plans | Medium | Medium |
| Block outline panel | Medium | Medium |
| Global user preferences | Low | Medium |
### Tier 3: Nice to Have
| Item | Effort | Impact |
|------|--------|--------|
| Inline refinement popover | High | Medium |
| Streaming preview pane | High | Medium |
| Smart suggestions bar | Medium | Low |
| Full-auto mode | High | Low |
---
## Part 7: Conclusion
### What's Already Agentic ✅
1. **@mention system** - Best-in-class block targeting
2. **Slash commands** - IDE-like quick actions
3. **Clarification quiz** - Proactive context gathering
4. **Edit plan preview** - Shows intent before action
5. **Section tracking** - Maintains document structure
### What's Missing for True Agentic ❌
1. **Approval workflow** - Changes should be reviewable
2. **Undo/history** - Need to revert AI mistakes
3. **Autonomous execution** - Agent should complete tasks independently
4. **Learning/memory** - Should improve over time
5. **Keyboard-first UX** - Power users need shortcuts
### Final Recommendation
The plugin has **strong agentic bones** but needs the **safety net** features that make IDEs trustworthy:
1. **Immediate wins:** Keyboard shortcuts + Undo button
2. **Medium-term:** Accept/Reject workflow + Command palette
3. **Long-term:** Autonomous mode + Learning system
The goal should be: *"I can trust this agent to make changes because I can always review, approve, or revert."*
---
## Part 8: Proactive AI Suggestions (NEW REQUIREMENT)
### 8.1 Current State
**Problem:** Agent only responds to commands - purely reactive.
### 8.2 Target State
**Goal:** Agent proactively analyzes content and suggests improvements.
### 8.3 Implementation Specification
#### 8.3.1 Idle Analysis Trigger
```javascript
// Trigger analysis after user stops editing for N seconds
const IDLE_THRESHOLD_MS = 5000; // 5 seconds
let idleTimer = null;
const startIdleAnalysis = () => {
clearTimeout(idleTimer);
idleTimer = setTimeout(() => {
analyzeArticleForSuggestions();
}, IDLE_THRESHOLD_MS);
};
// Hook into editor changes
wp.data.subscribe(() => {
startIdleAnalysis();
});
```
#### 8.3.2 Suggestion Types
| Category | Example Suggestion |
|----------|--------------------|
| **Clarity** | "Paragraph 3 could be clearer. Want me to simplify it?" |
| **Flow** | "The transition between sections 2 and 3 feels abrupt." |
| **Depth** | "This section could use more examples or data." |
| **SEO** | "Consider adding keyword 'X' to heading 2." |
| **Structure** | "This article could benefit from a summary section." |
| **Engagement** | "Consider adding a question to engage readers here." |
#### 8.3.3 UI Component
```
┌─────────────────────────────────────────────────┐
│ 💡 Suggestion │
│ │
│ "I noticed paragraph 3 could be clearer. │
│ Want me to refine it?" │
│ │
│ [Apply] [Dismiss] [Don't show again] │
└─────────────────────────────────────────────────┘
```
#### 8.3.4 Backend Endpoint
```php
// New REST endpoint: /analyze-for-suggestions
public function analyze_for_suggestions() {
$blocks = $this->get_all_blocks();
$prompt = "Analyze this article and suggest 1-3 improvements...";
// Return structured suggestions
}
```
#### 8.3.5 User Preferences
- Toggle: "Enable proactive suggestions"
- Frequency: "Aggressive / Balanced / Minimal"
- Categories: Checkboxes for which suggestion types to show
---
## Part 9: SEO Specialist Capabilities (NEW REQUIREMENT)
### 9.1 Vision
Agentic Writer should not only write well but write **SEO-optimized content** that ranks.
### 9.2 SEO Feature Matrix
| Feature | Description | Priority |
|---------|-------------|----------|
| **Keyword Analysis** | Analyze target keyword, suggest density | High |
| **Keyword Placement** | Ensure keyword in title, H1, first paragraph | High |
| **Heading Structure** | Validate H1→H2→H3 hierarchy | High |
| **Meta Generation** | Auto-generate meta title & description | High |
| **Readability Score** | Flesch-Kincaid or similar | Medium |
| **Internal Linking** | Suggest links to other posts | Medium |
| **Image Alt Text** | Auto-generate SEO-friendly alt text | Medium |
| **Schema Markup** | Suggest FAQ, HowTo, Article schema | Medium |
| **Competitor Analysis** | Compare with top-ranking articles | Low |
| **SERP Preview** | Show how it will appear in Google | Low |
### 9.3 SEO Workflow Integration
#### 9.3.1 Pre-Writing Phase
```
┌─────────────────────────────────────────────────┐
│ 🎯 SEO Setup │
│ │
│ Target Keyword: [_______________] │
│ Secondary Keywords: [_______________] │
│ │
│ ☑ Analyze competition before writing │
│ ☑ Include keyword in title │
│ ☑ Suggest internal links │
│ │
│ [Analyze Competition] [Skip to Writing] │
└─────────────────────────────────────────────────┘
```
#### 9.3.2 During Writing (Real-time)
- **Keyword density indicator** in sidebar
- **Heading structure validator**
- **Reading time & word count**
- **Readability score (live)**
#### 9.3.3 Post-Writing (SEO Audit)
```
┌─────────────────────────────────────────────────┐
│ 📊 SEO Score: 78/100 │
│ │
│ ✅ Keyword in title │
│ ✅ Keyword in first paragraph │
│ ⚠️ Keyword density low (0.8%, target 1-2%) │
│ ❌ Missing meta description │
│ ❌ No internal links found │
│ ✅ Proper heading hierarchy │
│ ⚠️ Images missing alt text (2 of 3) │
│ │
│ [Fix All Issues] [Generate Meta] [Add Links] │
└─────────────────────────────────────────────────┘
```
### 9.4 SEO-Aware System Prompts
Modify the plan generation prompt to include SEO considerations:
```
You are an SEO-optimized content writer. When creating content:
1. KEYWORD PLACEMENT:
- Include target keyword in H1 and first 100 words
- Use keyword naturally 1-2% density
- Include semantic variations
2. STRUCTURE:
- Use proper heading hierarchy (H1→H2→H3)
- Include table of contents for long articles
- Use bullet points and numbered lists
- Keep paragraphs under 150 words
3. ENGAGEMENT:
- Start with a hook
- Use questions to engage readers
- Include actionable takeaways
4. TECHNICAL:
- Suggest descriptive image alt text
- Recommend internal link opportunities
- Optimize for featured snippets where applicable
```
### 9.5 Post Config Additions
```javascript
const seoConfig = {
target_keyword: '',
secondary_keywords: [],
enable_seo_mode: true,
keyword_density_target: 1.5, // percentage
min_word_count: 1500,
include_meta: true,
include_schema: false,
internal_linking: true,
};
```
### 9.6 New REST Endpoints
| Endpoint | Purpose |
|----------|--------|
| `/seo/analyze-keyword` | Get keyword difficulty, volume |
| `/seo/audit-content` | Full SEO audit of current content |
| `/seo/generate-meta` | Generate meta title/description |
| `/seo/suggest-links` | Find internal linking opportunities |
| `/seo/competitor-analysis` | Analyze top-ranking content |
---
## Part 10: AI Model Recommendations (NEW REQUIREMENT)
### 10.1 Purpose
Help users choose the **cheapest AND best** models for agentic workflows.
### 10.2 Model Tiers by Use Case
#### 10.2.1 Planning Phase (Fast, Cheap)
| Model | Cost (per 1M tokens) | Speed | Quality | Recommendation |
|-------|---------------------|-------|---------|----------------|
| `google/gemini-2.0-flash-exp` | ~$0.10 in / $0.40 out | ⚡ Very Fast | Good | **Best Value** |
| `google/gemini-flash-1.5` | ~$0.075 in / $0.30 out | ⚡ Very Fast | Good | Budget Option |
| `anthropic/claude-3-haiku` | ~$0.25 in / $1.25 out | Fast | Good | Reliable |
| `openai/gpt-4o-mini` | ~$0.15 in / $0.60 out | Fast | Good | Alternative |
#### 10.2.2 Execution Phase (Quality Critical)
| Model | Cost (per 1M tokens) | Speed | Quality | Recommendation |
|-------|---------------------|-------|---------|----------------|
| `anthropic/claude-sonnet-4` | ~$3 in / $15 out | Medium | Excellent | **Best for Writing** |
| `anthropic/claude-3.5-sonnet` | ~$3 in / $15 out | Medium | Excellent | Proven Quality |
| `openai/gpt-4o` | ~$2.50 in / $10 out | Medium | Excellent | Alternative |
| `google/gemini-pro-1.5` | ~$1.25 in / $5 out | Medium | Very Good | Cost-Conscious |
#### 10.2.3 Image Generation
| Model | Cost (per image) | Speed | Quality | Recommendation |
|-------|-----------------|-------|---------|----------------|
| `black-forest-labs/flux-schnell` | ~$0.003 | ⚡ Very Fast | Good | **Best Value** |
| `black-forest-labs/flux-1.1-pro` | ~$0.04 | Fast | Excellent | Premium |
| `openai/dall-e-3` | ~$0.04-0.08 | Medium | Excellent | Alternative |
### 10.3 Recommended Configurations
#### 10.3.1 Budget-Conscious Setup (~$0.05 per article)
```
Planning Model: google/gemini-2.0-flash-exp
Execution Model: google/gemini-pro-1.5
Image Model: black-forest-labs/flux-schnell
```
#### 10.3.2 Balanced Setup (~$0.15 per article)
```
Planning Model: google/gemini-2.0-flash-exp
Execution Model: anthropic/claude-3.5-sonnet
Image Model: black-forest-labs/flux-schnell
```
#### 10.3.3 Premium Quality Setup (~$0.30+ per article)
```
Planning Model: anthropic/claude-3-haiku
Execution Model: anthropic/claude-sonnet-4
Image Model: black-forest-labs/flux-1.1-pro
```
### 10.4 Model Selection UI
```
┌─────────────────────────────────────────────────┐
│ 🤖 Model Configuration │
│ │
│ Preset: [Budget ▾] [Balanced ▾] [Premium ▾] │
│ │
│ Planning Model: │
│ [google/gemini-2.0-flash-exp ▾] │
│ 💰 ~$0.10/1M tokens • ⚡ Fast │
│ │
│ Execution Model: │
│ [anthropic/claude-sonnet-4 ▾] │
│ 💰 ~$3/1M tokens • ✨ Best Quality │
│ │
│ Image Model: │
│ [black-forest-labs/flux-schnell ▾] │
│ 💰 ~$0.003/image • ⚡ Fast │
│ │
│ Estimated cost per article: ~$0.15 │
└─────────────────────────────────────────────────┘
```
### 10.5 Smart Model Suggestions
The plugin should analyze usage patterns and suggest:
1. **If budget is tight:**
> "You've spent $45 this month. Switch to Budget preset to save ~60%."
2. **If quality issues detected:**
> "Multiple refinements on recent articles. Consider upgrading execution model."
3. **If speed is priority:**
> "For faster generation, switch planning to Gemini Flash."
---
## Part 11: Updated Roadmap
### Phase 1: Complete Tier 1 ✅ COMPLETED
- [x] Cmd+Enter to send
- [x] Accept/Reject workflow (Apply/Cancel)
- [x] Diff preview for edit plans
- [x] **Undo for AI changes** (sidebar.js: aiUndoStack, pushUndoSnapshot, undoLastAiOperation)
- [x] **Budget tracker enhanced** (Cost tab with refresh, remaining display, warnings)
- [x] **Settings page revamp** (Modern card-based UI with preset configurations)
### Phase 2: SEO Foundation ✅ COMPLETED
- [x] Add SEO config fields (focus keyword, secondary keywords, meta description)
- [x] Integrate SEO considerations into system prompts (build_seo_context)
- [x] Add keyword density indicator (in SEO audit)
- [x] Add SEO audit endpoint (/seo-audit/{post_id})
- [x] Add meta generation (/generate-meta with AI)
### Phase 3: Proactive Suggestions (2-3 weeks)
- [ ] Implement idle detection
- [ ] Create suggestion analysis endpoint
- [ ] Add suggestion UI component
- [ ] Add user preferences for suggestions
### Phase 4: Model Recommendations ✅ COMPLETED
- [x] Add preset configurations (Budget, Balanced, Premium)
- [x] Add model selection UI with cost estimates
- [ ] Add smart suggestions based on usage
### Phase 5: Tier 2 Features (3-4 weeks)
- [ ] Command palette (Cmd+Shift+P)
- [ ] Per-action accept/reject in plans
- [ ] Block outline panel
- [ ] Global user preferences
---
**Report Updated:** January 22, 2026
**Implementation Status:** Phase 1, Phase 2 (SEO Foundation) & Phase 4 completed. Ready for Phase 3 (Proactive Suggestions).

786
docs/architecture/AGENTIC_CONTEXT_STRATEGY.md Normal file
View File

@@ -0,0 +1,786 @@
# Agentic Context Management Strategy
**Date:** January 25, 2026
**Version:** 0.1.3+
**Purpose:** AI-powered context management for multilingual, intelligent user experience
---
## 🎯 Core Philosophy: Let AI Handle AI Context
### **The Problem with Hardcoded Solutions**
**Previous Approach (FLAWED):**
```javascript
// ❌ English-only, brittle, not scalable
if (content.includes('outline') || content.includes('structure')) {
return 'create_outline';
}
```
**Issues:**
- ❌ Only works in English
- ❌ Breaks in Indonesian, Arabic, Chinese, etc.
- ❌ Misses nuanced intent
- ❌ Requires constant maintenance
- ❌ Goes against "agentic" philosophy
### **Agentic Principle**
> **"If AI is smart enough to write articles, it's smart enough to manage its own context."**
**New Approach:**
- ✅ Use AI to summarize chat history
- ✅ Use AI to detect user intent
- ✅ Language-agnostic (works in any language)
- ✅ Adapts to context automatically
- ✅ True "agentic" experience
---
## 💰 Cost Analysis: AI-Powered Context Management
### **Your Cost Reference**
```
Action: meta_description
Model: deepseek-chat-v3-032
Tokens: 510
Cost: $0.0001
```
**This is EXTREMELY cheap!** Let's use this model for context operations.
### **Proposed Actions**
#### **1. Action: `summarize_context`**
**Purpose:** Condense long chat history into key points
**Input:**
```json
{
"action": "summarize_context",
"chat_history": [
{"role": "user", "content": "Saya ingin menulis tentang keamanan WordPress"},
{"role": "assistant", "content": "[Long response in Indonesian...]"},
{"role": "user", "content": "Fokus pada plugin vulnerabilities saja"},
{"role": "assistant", "content": "[Detailed plugin security response...]"}
]
}
```
**Prompt:**
```
Summarize this conversation into key points that capture the user's intent and requirements.
Focus on:
- Main topic
- Specific focus areas
- Rejected/excluded topics
- User preferences (tone, audience, etc.)
Keep the summary concise (max 200 words) but preserve critical context.
Write in the same language as the conversation.
Output format:
TOPIC: [main topic]
FOCUS: [what to include]
EXCLUDE: [what to avoid]
PREFERENCES: [any specific requirements]
```
**Expected Output:**
```
TOPIC: WordPress security
FOCUS: Plugin vulnerabilities only
EXCLUDE: Performance optimization, backup strategies (user rejected these)
PREFERENCES: Technical audience, detailed explanations
```
**Cost Estimate:**
- Input: 4,000 tokens (long chat history)
- Output: 100 tokens (summary)
- Model: deepseek-chat-v3-032
- **Cost: ~$0.0001 per summarization**
**When to Use:**
- Chat history > 6 messages
- Before generating outline
- Before executing article
---
#### **2. Action: `detect_intent`**
**Purpose:** Understand what user wants to do next
**Input:**
```json
{
"action": "detect_intent",
"last_message": "Baiklah, sekarang buatkan outline-nya",
"has_plan": false,
"current_mode": "chat"
}
```
**Prompt:**
```
Based on the user's message, determine their intent. Choose ONE:
1. "create_outline" - User wants to create an article outline/structure
2. "start_writing" - User wants to write the full article
3. "refine_content" - User wants to improve existing content
4. "continue_chat" - User wants to continue discussing/exploring
5. "clarify" - User is asking questions or needs clarification
Consider:
- The user's explicit request
- Whether they have an outline already (has_plan: {has_plan})
- Current mode (current_mode: {current_mode})
Respond with ONLY the intent code (e.g., "create_outline").
```
**Expected Output:**
```
create_outline
```
**Cost Estimate:**
- Input: 100 tokens (last message + context)
- Output: 5 tokens (intent code)
- Model: deepseek-chat-v3-032
- **Cost: ~$0.00002 per detection**
**When to Use:**
- After every user message in Chat mode
- To show contextual action buttons
- To auto-suggest next steps
---
## 📊 Cost Comparison: Full History vs AI-Powered
### **Scenario: 5 Agent + 4 Human Messages**
| Approach | Input Tokens | Output Tokens | Cost per Request | Quality | Language Support |
|----------|--------------|---------------|------------------|---------|------------------|
| **Full History** | 4,365 | 0 | $0.013 (Claude) | ✅ Best | ✅ All |
| **AI Summarization** | 100 (summary) | 0 | $0.003 (Claude) + $0.0001 (summary) | ✅ Good | ✅ All |
| **Hardcoded Pruning** | 1,800 | 0 | $0.005 (Claude) | ⚠️ Fair | ❌ English only |
| **No Context** | 0 | 0 | $0.000 | ❌ Poor | ✅ All |
### **Cost Breakdown for 100 Articles/Month**
**Full History:**
- Planning: 100 × $0.00033 = $0.033
- Execution: 100 × $0.013 = $1.30
- **Total: $1.33/month**
**AI Summarization:**
- Summarization: 100 × $0.0001 = $0.01
- Planning: 100 × $0.00008 = $0.008
- Execution: 100 × $0.003 = $0.30
- **Total: $0.32/month**
- **Savings: $1.01/month (76% reduction)**
**Intent Detection:**
- Per message: $0.00002
- Average 10 messages per article: 100 × 10 × $0.00002 = $0.02
- **Total: $0.02/month (negligible)**
---
## 🎯 The Big Picture: Agentic Experience
### **User Journey Analysis**
**Current Flow (Fragmented):**
```
1. User opens editor
2. User manually switches to Chat mode
3. User types message
4. Agent responds
5. User types more
6. Agent responds
7. User manually switches to Planning mode
8. User types "create outline"
9. Outline generated
10. User manually clicks "Start Writing"
11. Article generated
```
**Problems:**
- Too many manual mode switches
- User must know when to switch
- No guidance on next steps
- Friction in workflow
---
### **Proposed Agentic Flow (Seamless):**
```
1. User opens editor (any mode)
2. User types: "Saya ingin menulis tentang keamanan WordPress"
3. Agent responds with suggestions
4. User types: "Fokus pada plugin vulnerabilities"
5. Agent responds with refined ideas
6. 💡 UI shows: [📝 Ready to create outline?] (AI-detected intent)
7. User clicks button (or types "yes" or "buatkan outline")
8. ✨ AI summarizes chat history (0.1 seconds)
9. Outline generated with clean context
10. 💡 UI shows: [✍️ Start Writing] (auto-suggested)
11. User clicks
12. Article generated
```
**Improvements:**
- ✅ No manual mode switching needed
- ✅ AI suggests next steps proactively
- ✅ Context automatically optimized
- ✅ Smooth, guided experience
- ✅ Works in any language
---
## 🔧 Implementation Design
### **Backend: New Actions**
```php
/**
* Handle context summarization request.
*
* @param WP_REST_Request $request REST request.
* @return WP_REST_Response|WP_Error Response.
*/
public function handle_summarize_context( $request ) {
$params = $request->get_json_params();
$chat_history = $params['chatHistory'] ?? array();
if ( empty( $chat_history ) || count( $chat_history ) < 4 ) {
// No need to summarize short history
return new WP_REST_Response(
array(
'summary' => '',
'use_full_history' => true,
),
200
);
}
// Build summarization prompt
$history_text = '';
foreach ( $chat_history as $msg ) {
$role = ucfirst( $msg['role'] ?? 'Unknown' );
$content = $msg['content'] ?? '';
$history_text .= "{$role}: {$content}\n\n";
}
$prompt = "Summarize this conversation into key points that capture the user's intent and requirements.
Focus on:
- Main topic
- Specific focus areas
- Rejected/excluded topics
- User preferences (tone, audience, etc.)
Keep the summary concise (max 200 words) but preserve critical context.
Write in the same language as the conversation.
Output format:
TOPIC: [main topic]
FOCUS: [what to include]
EXCLUDE: [what to avoid]
PREFERENCES: [any specific requirements]
Conversation:
{$history_text}";
$provider = WP_Agentic_Writer_OpenRouter_Provider::get_instance();
$messages = array(
array(
'role' => 'user',
'content' => $prompt,
),
);
// Use cheap model for summarization
$response = $provider->chat( $messages, array(), 'summarize' );
if ( is_wp_error( $response ) ) {
return $response;
}
// Track cost
do_action(
'wp_aw_after_api_request',
$params['postId'] ?? 0,
$response['model'] ?? '',
'summarize_context',
$response['input_tokens'] ?? 0,
$response['output_tokens'] ?? 0,
$response['cost'] ?? 0
);
return new WP_REST_Response(
array(
'summary' => $response['content'] ?? '',
'use_full_history' => false,
'cost' => $response['cost'] ?? 0,
),
200
);
}
/**
* Handle intent detection request.
*
* @param WP_REST_Request $request REST request.
* @return WP_REST_Response|WP_Error Response.
*/
public function handle_detect_intent( $request ) {
$params = $request->get_json_params();
$last_message = $params['lastMessage'] ?? '';
$has_plan = $params['hasPlan'] ?? false;
$current_mode = $params['currentMode'] ?? 'chat';
if ( empty( $last_message ) ) {
return new WP_REST_Response(
array( 'intent' => 'continue_chat' ),
200
);
}
$prompt = "Based on the user's message, determine their intent. Choose ONE:
1. \"create_outline\" - User wants to create an article outline/structure
2. \"start_writing\" - User wants to write the full article
3. \"refine_content\" - User wants to improve existing content
4. \"continue_chat\" - User wants to continue discussing/exploring
5. \"clarify\" - User is asking questions or needs clarification
Consider:
- The user's explicit request
- Whether they have an outline already (has_plan: " . ( $has_plan ? 'true' : 'false' ) . ")
- Current mode (current_mode: {$current_mode})
User's message: \"{$last_message}\"
Respond with ONLY the intent code (e.g., \"create_outline\").";
$provider = WP_Agentic_Writer_OpenRouter_Provider::get_instance();
$messages = array(
array(
'role' => 'user',
'content' => $prompt,
),
);
$response = $provider->chat( $messages, array(), 'intent_detection' );
if ( is_wp_error( $response ) ) {
return $response;
}
// Track cost
do_action(
'wp_aw_after_api_request',
$params['postId'] ?? 0,
$response['model'] ?? '',
'detect_intent',
$response['input_tokens'] ?? 0,
$response['output_tokens'] ?? 0,
$response['cost'] ?? 0
);
$intent = trim( strtolower( $response['content'] ?? 'continue_chat' ) );
return new WP_REST_Response(
array(
'intent' => $intent,
'cost' => $response['cost'] ?? 0,
),
200
);
}
```
---
### **Frontend: Agentic UX**
```javascript
// Auto-detect intent after each message
const handleMessageSent = async (userMessage) => {
// Send message to chat
const chatResponse = await sendChatMessage(userMessage);
// Detect intent in background
const intentResponse = await fetch('/wp-json/wp-agentic-writer/v1/detect-intent', {
method: 'POST',
body: JSON.stringify({
lastMessage: userMessage,
hasPlan: !!currentPlan,
currentMode: agentMode,
postId: postId
})
});
const { intent } = await intentResponse.json();
// Show contextual action based on intent
setDetectedIntent(intent);
showContextualAction(intent);
};
// Render contextual action buttons
const renderContextualAction = () => {
if (!detectedIntent) return null;
switch (detectedIntent) {
case 'create_outline':
return (
<div className="contextual-action">
<p>💡 Ready to create an outline?</p>
<button onClick={handleCreateOutlineWithSummary} className="primary">
📝 Create Outline
</button>
</div>
);
case 'start_writing':
if (!currentPlan) {
return (
<div className="contextual-action">
<p>⚠️ You need an outline first</p>
<button onClick={handleCreateOutlineWithSummary}>
📝 Create Outline First
</button>
</div>
);
}
return (
<div className="contextual-action">
<p>💡 Ready to write the article?</p>
<button onClick={handleStartWriting} className="primary">
✍️ Start Writing
</button>
</div>
);
case 'refine_content':
return (
<div className="contextual-action">
<p>💡 Use @block to refine specific sections</p>
</div>
);
default:
return null;
}
};
// Create outline with AI summarization
const handleCreateOutlineWithSummary = async () => {
setIsLoading(true);
// Step 1: Summarize chat history if needed
let contextToSend = messages;
if (messages.length > 6) {
showStatus('Optimizing context...');
const summaryResponse = await fetch('/wp-json/wp-agentic-writer/v1/summarize-context', {
method: 'POST',
body: JSON.stringify({
chatHistory: messages,
postId: postId
})
});
const { summary, use_full_history, cost } = await summaryResponse.json();
if (!use_full_history && summary) {
// Use summarized context
contextToSend = [
{
role: 'system',
content: `Context Summary:\n${summary}`
},
...messages.slice(-2) // Keep last exchange
];
console.log('Context optimized. Cost:', cost);
}
}
// Step 2: Generate outline with optimized context
showStatus('Creating outline...');
const outlineResponse = await fetch('/wp-json/wp-agentic-writer/v1/generate-plan', {
method: 'POST',
body: JSON.stringify({
topic: extractTopic(messages),
chatHistory: contextToSend,
postId: postId,
postConfig: postConfig,
stream: true
})
});
// Handle streaming response...
};
```
---
## 🎨 UX Enhancements
### **1. Contextual Action Cards**
```
┌─────────────────────────────────────────────────────┐
│ Agent: "I can help you create a comprehensive │
│ outline for WordPress plugin security..." │
├─────────────────────────────────────────────────────┤
│ 💡 Detected Intent: Create Outline │
│ │
│ [📝 Create Outline] [💬 Continue Discussing] │
│ │
│ 💰 Context will be optimized (~$0.0001) │
└─────────────────────────────────────────────────────┘
```
### **2. Context Optimization Indicator**
```
┌─────────────────────────────────────────────────────┐
│ ⚡ Optimizing context... │
│ • 9 messages → Summary (200 words) │
│ • Token reduction: 4,365 → 450 (90%) │
│ • Cost: $0.0001 │
│ ✓ Done in 0.2s │
└─────────────────────────────────────────────────────┘
```
### **3. Smart Mode Transitions**
```
User in Chat mode types: "buatkan outline-nya"
┌─────────────────────────────────────────────────────┐
│ 💡 Switching to Planning mode... │
│ • Detected intent: Create outline │
│ • Optimizing 7 messages of context │
│ • Generating outline... │
└─────────────────────────────────────────────────────┘
[Outline appears]
┌─────────────────────────────────────────────────────┐
│ ✨ Outline ready! │
│ │
│ Next step: │
│ [✍️ Start Writing Article] │
└─────────────────────────────────────────────────────┘
```
---
## 📊 Decision Matrix: When to Use What?
| Situation | Recommended Approach | Reason |
|-----------|---------------------|--------|
| **Chat history ≤ 4 messages** | Send full history | Short enough, no optimization needed |
| **Chat history 5-8 messages** | AI summarization | Balance cost and quality |
| **Chat history > 8 messages** | AI summarization + last 2 | Keep recent context verbatim |
| **User switches modes** | Detect intent | Guide user to next action |
| **Before outline generation** | Summarize context | Clean, focused input |
| **Before article execution** | Use plan (no chat history) | Plan already has all context |
| **Block refinement** | No chat history | Block content is sufficient |
| **User types "/reset"** | Clear all context | Fresh start |
---
## 🎯 Recommendation: Hybrid Intelligent Approach
### **The Winning Strategy**
**Combine AI-powered + Smart Defaults:**
1. **Default Behavior (No User Action)**
- Chat history ≤ 4 messages → Send full history
- Chat history > 4 messages → Auto-summarize with AI
- Cost: ~$0.0001 per summarization (negligible)
2. **Intent Detection (Automatic)**
- After every user message → Detect intent
- Show contextual action buttons
- Cost: ~$0.00002 per detection (negligible)
3. **User Control (Optional)**
- Settings: "Context Mode" → Auto/Full/Minimal
- "/reset" command → Clear context
- Manual selection UI (advanced users)
### **Why This Works**
✅ **Language-Agnostic**
- Works in English, Indonesian, Arabic, Chinese, etc.
- No hardcoded keywords
✅ **Cost-Effective**
- 76% cost reduction vs full history
- Total added cost: ~$0.34/month for 100 articles
- ROI: Better quality + Lower cost
✅ **True Agentic Experience**
- AI manages its own context
- Proactive suggestions
- Seamless workflow
- No manual mode switching
✅ **User-Friendly**
- Automatic by default
- Optional manual control
- Transparent (shows what's happening)
- Fast (summarization takes 0.1-0.3s)
---
## 🔧 Implementation Plan
### **Phase 1: Core Infrastructure** (Week 1)
**Backend:**
- [ ] Add `/summarize-context` endpoint
- [ ] Add `/detect-intent` endpoint
- [ ] Add `summarize` and `intent_detection` operation types to cost tracking
- [ ] Update OpenRouter provider to support these actions
**Frontend:**
- [ ] Add `handleSummarizeContext()` function
- [ ] Add `handleDetectIntent()` function
- [ ] Add context optimization indicator component
**Testing:**
- [ ] Test summarization in English, Indonesian, Arabic
- [ ] Test intent detection in multiple languages
- [ ] Verify cost tracking
### **Phase 2: UX Integration** (Week 2)
**Frontend:**
- [ ] Add contextual action cards
- [ ] Auto-detect intent after each message
- [ ] Show "Optimizing context..." status
- [ ] Add smart mode transitions
**Settings:**
- [ ] Add "Context Mode" setting (Auto/Full/Minimal)
- [ ] Add context optimization toggle
- [ ] Add cost estimates in settings
**Testing:**
- [ ] Test full user journey (chat → outline → write)
- [ ] Test in multiple languages
- [ ] Verify smooth transitions
### **Phase 3: Advanced Features** (Week 3)
**Features:**
- [ ] Add `/reset` command
- [ ] Add manual context selection UI (optional)
- [ ] Add context analytics (token usage, cost breakdown)
- [ ] Add context caching (reuse summaries)
**Optimization:**
- [ ] Implement smart caching for summaries
- [ ] Add context relevance scoring
- [ ] Optimize prompt templates
**Documentation:**
- [ ] Update user guide
- [ ] Add context management section
- [ ] Document cost implications
---
## 💰 Final Cost Analysis
### **Per Article (Average)**
| Component | Cost | Frequency |
|-----------|------|-----------|
| Intent detection | $0.00002 × 10 messages | = $0.0002 |
| Context summarization | $0.0001 × 1 time | = $0.0001 |
| Planning (with summary) | $0.003 | = $0.003 |
| Execution (no history) | $0.50-$2.00 | = $1.00 avg |
| **Total per article** | | **≈ $1.0033** |
**Compared to Full History:**
- Full history approach: $1.013 per article
- AI-powered approach: $1.0033 per article
- **Savings: $0.01 per article** (negligible)
**But wait - the real benefit:**
- ✅ Better quality (clean, focused context)
- ✅ Language-agnostic (works everywhere)
- ✅ Better UX (proactive suggestions)
- ✅ Scalable (no hardcoded rules)
---
## 🎯 Answer to Your Question
### **"Send all chat history vs AI summarization?"**
**Answer: AI Summarization is Better**
**Reasons:**
1. **Cost is Nearly Identical**
- Full history: $1.013/article
- AI summary: $1.0033/article
- Difference: $0.01 (1% savings)
2. **Quality is Better**
- Summary removes contradicted ideas
- Summary focuses on final intent
- Summary prevents pollution
- AI explicitly told what to focus on
3. **Language Support**
- Full history: Works in all languages ✅
- AI summary: Works in all languages ✅
- Hardcoded: Only English ❌
4. **Agentic Experience**
- AI managing AI context = true agentic
- Proactive intent detection
- Seamless workflow
- No user friction
5. **Scalability**
- No hardcoded rules to maintain
- Adapts to new languages automatically
- Handles edge cases gracefully
### **The Plan:**
1. ✅ **Implement AI summarization** (not hardcoded)
2. ✅ **Implement AI intent detection** (not hardcoded)
3. ✅ **Make it automatic** (no user action needed)
4. ✅ **Add user controls** (optional override)
5. ✅ **Track costs transparently** (show user what's happening)
---
**Status:** 🚀 READY TO IMPLEMENT
**Approach:** AI-Powered (Agentic)
**Cost Impact:** Negligible (+$0.34/month for 100 articles)
**Quality Impact:** Significant improvement
**UX Impact:** Seamless, guided experience

741
docs/architecture/CONTEXT_FLOW_ANALYSIS.md Normal file
View File

@@ -0,0 +1,741 @@
# Context Flow Analysis: Chat, Planning, and Writing Modes
**Date:** January 25, 2026
**Version:** 0.1.3+
**Purpose:** Comprehensive analysis of context preservation across different user interaction flows
---
## 🎯 Executive Summary
This document analyzes how context (chat history, post configuration, language detection, and plan data) is preserved or lost across different user interaction flows in the WP Agentic Writer plugin. It identifies **12 distinct user flows**, maps their context behavior, and provides recommendations for improvements.
**Key Finding:** Context preservation varies significantly depending on the flow path, with some flows maintaining full context while others may lose critical information.
---
## 📊 Context Storage Mechanisms
### **1. Post Meta Storage**
The plugin stores context in WordPress post meta:
| Meta Key | Content | Updated By | Used By |
|----------|---------|------------|---------|
| `_wpaw_chat_history` | Array of user/assistant messages | `update_post_chat_history()` | Chat mode, Plan generation |
| `_wpaw_plan` | JSON outline structure | `stream_generate_plan()` | Article execution |
| `_wpaw_detected_language` | Language code (e.g., 'Indonesian') | `stream_generate_plan()` | All modes |
| `_wpaw_post_config` | Post configuration (tone, audience, SEO) | `update_post_config()` | All modes |
| `_wpaw_memory` | Last prompt/intent tracking | `update_post_memory()` | Internal tracking |
### **2. Frontend State (React)**
| State Variable | Scope | Persistence |
|----------------|-------|-------------|
| `messages` | Component state | Session only (lost on refresh) |
| `agentMode` | localStorage | Persists across sessions |
| `postConfig` | Component state | Session only |
| `currentPlan` | useRef | Session only |
### **3. Request Parameters**
Context is passed via REST API requests:
- `chatHistory` - Array of messages from frontend
- `postConfig` - Current configuration
- `detectedLanguage` - Language from clarity quiz
- `clarificationAnswers` - Answers from clarity quiz
---
## 🔄 User Interaction Flows
### **Flow 1: Standard Flow (Chat → Planning → Writing)**
**Path:** Chat mode → Ask for outline → Planning mode generates plan → Click "Start Writing" → Writing mode executes
**Context Behavior:**
```
1. User types in Chat mode
├─ Messages stored in frontend state
├─ Chat history saved to post meta (_wpaw_chat_history)
└─ Language detected and stored (_wpaw_detected_language)
2. Planning mode generates outline
├─ Receives: chatHistory from frontend
├─ Builds: chat_history_context from chatHistory array
├─ Stores: Plan in _wpaw_plan
├─ Stores: Language in _wpaw_detected_language
└─ Keywords auto-suggested (if SEO enabled)
3. Click "Start Writing"
├─ Reads: _wpaw_plan (required)
├─ Reads: _wpaw_detected_language
├─ Reads: postConfig from frontend
└─ Generates article with full context
```
**Context Preservation:** ✅ **EXCELLENT**
- Chat history: ✅ Available via chatHistory parameter
- Plan: ✅ Stored in post meta
- Language: ✅ Stored in post meta
- Post config: ✅ Passed from frontend
**Potential Issues:** None - This is the ideal flow.
---
### **Flow 2: Direct Writing Mode (No Chat, No Planning)**
**Path:** Switch to Writing mode → Type instruction → Send
**Context Behavior:**
```
1. User switches to Writing mode
├─ Frontend: agentMode = 'writing'
└─ No chat history built
2. User types instruction
├─ Frontend sends to /chat endpoint with type='writing'
├─ Backend: handle_chat_request()
├─ No plan required for chat endpoint
└─ Response returned
3. If user wants to generate article
├─ Must have a plan (_wpaw_plan)
└─ ERROR: "No plan found. Please generate a plan first."
```
**Context Preservation:** ⚠️ **LIMITED**
- Chat history: ✅ Can be built from messages
- Plan: ❌ **NOT AVAILABLE** - Cannot execute article
- Language: ⚠️ May not be detected
- Post config: ✅ Available from frontend
**Issues:**
1. **Cannot execute article without plan** - Writing mode chat doesn't create a plan
2. **User confusion** - Writing mode suggests article generation but can't deliver
3. **No outline context** - AI has no structure to follow
**Recommendation:**
- Writing mode should either:
- A) Auto-generate a minimal plan from the instruction, OR
- B) Show clear error: "Please create an outline first (switch to Planning mode)"
---
### **Flow 3: Chat → Direct Writing (Skip Planning)**
**Path:** Chat mode → Build context → Switch to Writing mode → Type instruction → Send
**Context Behavior:**
```
1. User chats in Chat mode
├─ Chat history built in frontend state
├─ Saved to _wpaw_chat_history
└─ Language may be detected
2. User switches to Writing mode
├─ Frontend: agentMode = 'writing'
└─ Chat history still in frontend state
3. User types instruction in Writing mode
├─ Sends to /chat endpoint with type='writing'
├─ chatHistory parameter: ⚠️ NOT SENT (only sent to /generate-plan)
├─ Backend has no access to previous chat
└─ Response generated without chat context
4. If user tries to execute article
└─ ERROR: "No plan found"
```
**Context Preservation:** ❌ **POOR**
- Chat history: ❌ **LOST** - Not sent to /chat endpoint in writing mode
- Plan: ❌ Not available
- Language: ⚠️ May be stored from earlier chat
- Post config: ✅ Available
**Issues:**
1. **Chat context lost** - Previous conversation not available to AI
2. **No plan** - Cannot execute article
3. **Inconsistent behavior** - User expects context to carry over
**Recommendation:**
- Send `chatHistory` parameter to `/chat` endpoint for all modes
- OR retrieve `_wpaw_chat_history` from post meta in backend
---
### **Flow 4: Planning → Manual Mode Switch → Writing**
**Path:** Planning mode → Generate outline → Manually switch to Writing → Add note → Send
**Context Behavior:**
```
1. Planning mode generates outline
├─ Plan stored in _wpaw_plan
├─ Language stored
└─ Chat history stored
2. User manually switches to Writing mode
├─ Frontend: agentMode = 'writing'
└─ Plan still in post meta
3. User adds additional note
├─ Sends to /chat endpoint with type='writing'
├─ chatHistory: ⚠️ NOT SENT
├─ AI responds without knowing about the plan
└─ User's note not incorporated into plan
4. User tries to execute article
├─ Reads _wpaw_plan (original plan, unchanged)
├─ User's additional note: ❌ NOT INCLUDED
└─ Article generated from original plan only
```
**Context Preservation:** ⚠️ **PARTIAL**
- Chat history: ❌ Not sent to writing mode chat
- Plan: ✅ Available but not updated
- Language: ✅ Available
- Post config: ✅ Available
- **User's additional note:** ❌ **LOST**
**Issues:**
1. **Additional instructions ignored** - User's note in writing mode doesn't update plan
2. **Confusing UX** - User expects their note to be incorporated
3. **No plan revision** - Writing mode chat doesn't trigger plan update
**Recommendation:**
- Writing mode should either:
- A) Update the plan with user's additional instructions, OR
- B) Store the note and append it to execution prompt, OR
- C) Show warning: "To modify the outline, switch back to Planning mode"
---
### **Flow 5: Direct Planning Mode (No Chat)**
**Path:** Switch to Planning mode → Type topic → Generate outline
**Context Behavior:**
```
1. User switches to Planning mode
└─ No prior chat history
2. User types topic
├─ Sends to /generate-plan endpoint
├─ chatHistory: [] (empty)
├─ chat_history_context: "" (empty)
└─ Plan generated from topic only
3. Plan generation
├─ Stores plan in _wpaw_plan
├─ Detects and stores language
└─ No chat history to reference
```
**Context Preservation:** ✅ **GOOD**
- Chat history: N/A (none exists)
- Plan: ✅ Generated and stored
- Language: ✅ Detected and stored
- Post config: ✅ Available
**Issues:** None - This is a valid flow for quick outline generation.
---
### **Flow 6: Chat → Planning with Clarity Quiz**
**Path:** Chat mode → Build context → Planning mode → Clarity quiz appears → Answer questions → Generate
**Context Behavior:**
```
1. User chats in Chat mode
├─ Chat history built
└─ Language detected
2. Planning mode triggers clarity quiz
├─ Frontend: setInClarification(true)
└─ Quiz questions shown
3. User answers quiz
├─ Answers stored in frontend state
└─ postConfig updated with answers
4. Submit quiz
├─ Sends to /generate-plan with:
│ ├─ clarificationAnswers
│ ├─ chatHistory ✅
│ ├─ postConfig ✅
│ └─ detectedLanguage ✅
└─ Plan generated with full context
```
**Context Preservation:** ✅ **EXCELLENT**
- Chat history: ✅ Sent via chatHistory
- Clarity answers: ✅ Sent via clarificationAnswers
- Language: ✅ Sent via detectedLanguage
- Post config: ✅ Updated with quiz answers
**Issues:** None - This is the ideal flow with maximum context.
---
### **Flow 7: Writing Mode → @block Refinement**
**Path:** Article generated → Switch to Writing mode → Use @block mention → Refine specific block
**Context Behavior:**
```
1. Article already generated
├─ Content in Gutenberg blocks
└─ Plan in _wpaw_plan
2. User types "@block-id refine this section"
├─ Frontend detects @mention
├─ Sends to /refine-block endpoint
└─ NOT to /chat endpoint
3. Block refinement
├─ Receives: block content, refinement request
├─ Receives: articleContext (surrounding blocks)
├─ Receives: postConfig
├─ chatHistory: ❌ NOT SENT
└─ Refinement done without chat context
```
**Context Preservation:** ⚠️ **PARTIAL**
- Block content: ✅ Available
- Article context: ✅ Sent (surrounding blocks)
- Post config: ✅ Available
- Chat history: ❌ Not sent
- Original plan: ⚠️ Not explicitly sent
**Issues:**
1. **Chat context lost** - Previous conversation not available
2. **Original intent unclear** - AI doesn't know user's original goals
**Recommendation:**
- Include `chatHistory` or at least last few messages in refinement requests
- Include original plan section for context
---
### **Flow 8: Multiple Chat Sessions (Page Refresh)**
**Path:** Chat → Build context → Refresh page → Continue chatting
**Context Behavior:**
```
1. First session
├─ Chat history in frontend state
└─ Saved to _wpaw_chat_history post meta
2. Page refresh
├─ Frontend state: ❌ CLEARED
├─ Post meta: ✅ PERSISTS
└─ agentMode: ✅ Restored from localStorage
3. Continue chatting
├─ Frontend loads chat history from post meta
├─ Displays previous messages
├─ New messages appended
└─ Context maintained
```
**Context Preservation:** ✅ **GOOD**
- Chat history: ✅ Restored from post meta
- Plan: ✅ Persists in post meta
- Language: ✅ Persists in post meta
- Post config: ⚠️ May need to be re-fetched
**Issues:**
1. **Initial load delay** - Need to fetch chat history from backend
2. **Post config sync** - May not reflect latest changes immediately
**Recommendation:**
- Ensure chat history is loaded on component mount
- Fetch post config from backend on load
---
### **Flow 9: Plan Revision in Planning Mode**
**Path:** Planning mode → Generate outline → User asks for changes → Plan revised
**Context Behavior:**
```
1. Initial plan generated
├─ Plan stored in _wpaw_plan
└─ Displayed in frontend
2. User types revision request
├─ Frontend detects existing plan
├─ Calls revisePlanFromPrompt()
├─ Sends to /revise-plan endpoint
└─ NOT to /generate-plan
3. Plan revision
├─ Receives: instruction, current plan
├─ Receives: postConfig
├─ chatHistory: ⚠️ NOT EXPLICITLY SENT
├─ Generates revised plan
└─ Updates _wpaw_plan
```
**Context Preservation:** ✅ **GOOD**
- Current plan: ✅ Sent explicitly
- Post config: ✅ Available
- Chat history: ⚠️ Not sent but may not be needed
- Language: ✅ Available from post meta
**Issues:**
1. **Chat context not used** - Previous conversation not considered
2. **Revision-only context** - AI only sees current plan + new instruction
**Recommendation:**
- Consider sending recent chat history for better context
- OR document that plan revision is isolated from chat history
---
### **Flow 10: SEO Keyword Suggestion Flow**
**Path:** Planning mode → Generate outline → Keywords auto-suggested → Clarity quiz pre-filled
**Context Behavior:**
```
1. Outline generated
├─ Plan stored
└─ Frontend receives plan
2. Auto-trigger keyword suggestion
├─ Calls /suggest-keywords endpoint
├─ Sends: title, sections, language
├─ chatHistory: ❌ NOT SENT
└─ Keywords suggested based on outline only
3. Keywords returned
├─ Stored in frontend state
├─ Pre-filled in clarity quiz
└─ User can edit before submission
4. Clarity quiz submitted
├─ Keywords saved to postConfig
└─ Used in article generation
```
**Context Preservation:** ✅ **GOOD**
- Outline: ✅ Sent to keyword suggester
- Language: ✅ Sent
- Chat history: ❌ Not sent (not needed)
- Keywords: ✅ Stored in postConfig
**Issues:** None - Keywords are based on outline, which is sufficient context.
---
### **Flow 11: Web Search Integration**
**Path:** Chat/Planning with web search enabled → AI searches web → Results incorporated
**Context Behavior:**
```
1. User enables web search
├─ postConfig.web_search = true
└─ Passed to backend
2. Chat or plan generation
├─ Backend builds web_search_options
├─ Passes to OpenRouter API
└─ AI performs web search
3. Search results
├─ Incorporated into AI response
├─ NOT stored separately
└─ Included in chat history as part of response
4. Subsequent requests
├─ Search results: ⚠️ Only in chat history
└─ Not explicitly tracked
```
**Context Preservation:** ⚠️ **PARTIAL**
- Search results: ⚠️ Only in chat history text
- Search metadata: ❌ Not stored
- Search queries: ❌ Not logged
**Issues:**
1. **No search audit trail** - Can't see what was searched
2. **Search results ephemeral** - Lost if chat history cleared
**Recommendation:**
- Consider logging search queries and results
- Store search metadata for debugging/auditing
---
### **Flow 12: Multi-Block Batch Refinement**
**Path:** Article generated → Select multiple blocks → Request batch refinement
**Context Behavior:**
```
1. User selects multiple blocks
├─ Frontend: blocksToRefine array
└─ allBlocks for context
2. Batch refinement request
├─ Sends to /refine-blocks endpoint
├─ Receives: blocksToRefine, allBlocks, instruction
├─ Receives: postConfig
├─ chatHistory: ❌ NOT SENT
├─ Plan: ❌ NOT SENT
└─ Refinement based on blocks + instruction only
3. Refinement execution
├─ Each block refined individually
├─ Context: surrounding blocks
└─ No cross-block coordination
```
**Context Preservation:** ⚠️ **LIMITED**
- Block content: ✅ Available
- Surrounding context: ✅ Available (allBlocks)
- Post config: ✅ Available
- Chat history: ❌ Not sent
- Original plan: ❌ Not sent
- Cross-block coordination: ❌ Not implemented
**Issues:**
1. **No chat context** - Original conversation lost
2. **No plan context** - Original outline not referenced
3. **Independent refinements** - Blocks refined in isolation
**Recommendation:**
- Send original plan section for each block
- Consider chat history for understanding user intent
- Implement cross-block coordination for consistency
---
## 🔍 Context Loss Scenarios
### **Critical Context Loss Issues**
| Scenario | Lost Context | Impact | Severity |
|----------|--------------|--------|----------|
| Chat → Writing mode switch | Chat history not sent to /chat endpoint | AI doesn't know previous conversation | 🔴 HIGH |
| Writing mode additional notes | Notes not incorporated into plan | User instructions ignored | 🔴 HIGH |
| Block refinement | Chat history not available | Original intent unclear | 🟡 MEDIUM |
| Page refresh | Frontend state cleared | Need to reload from backend | 🟢 LOW |
| Web search results | Search metadata not stored | No audit trail | 🟢 LOW |
---
## 💡 Recommendations
### **Priority 1: Critical Fixes**
1. **Send Chat History to All Modes**
```javascript
// In sidebar.js - sendMessage function
const payload = {
messages: messages,
postId: postId,
type: agentMode,
chatHistory: messages, // ✅ Add this
postConfig: postConfig,
stream: true
};
```
2. **Handle Writing Mode Notes**
- Option A: Auto-update plan with additional instructions
- Option B: Store notes and append to execution prompt
- Option C: Show clear warning about mode limitations
3. **Improve Block Refinement Context**
```php
// In handle_refine_block
$chat_history = $this->get_post_chat_history( $post_id );
$plan = get_post_meta( $post_id, '_wpaw_plan', true );
// Include in refinement prompt
```
### **Priority 2: UX Improvements**
4. **Mode-Specific Guidance**
- Chat mode: "Building context for your article"
- Planning mode: "Creating outline - chat history will be used"
- Writing mode: "Refining content - outline required"
5. **Context Indicators**
- Show badge: "📝 Outline available"
- Show badge: "💬 3 messages in context"
- Show badge: "🔍 Web search enabled"
6. **Error Messages**
- Writing mode without plan: "Please create an outline first (switch to Planning mode)"
- Refinement without context: "Loading article context..."
### **Priority 3: Advanced Features**
7. **Context Persistence Layer**
```php
// Store comprehensive context
update_post_meta( $post_id, '_wpaw_context', array(
'chat_history' => $chat_history,
'plan' => $plan,
'language' => $language,
'config' => $post_config,
'search_history' => $search_queries,
'refinement_history' => $refinements,
));
```
8. **Context Debugging Tool**
- Admin panel showing current context state
- "What does the AI know?" button
- Context timeline visualization
9. **Smart Context Pruning**
- Keep last N messages (configurable)
- Summarize older context
- Preserve critical information (plan, config)
---
## 📋 Context Flow Matrix
| Flow | Chat History | Plan | Language | Post Config | Notes |
|------|--------------|------|----------|-------------|-------|
| Chat → Planning → Writing | ✅ Full | ✅ Full | ✅ Full | ✅ Full | **Ideal flow** |
| Direct Writing | ⚠️ Limited | ❌ None | ⚠️ Partial | ✅ Full | Cannot execute |
| Chat → Writing (skip plan) | ❌ Lost | ❌ None | ⚠️ Partial | ✅ Full | Context lost |
| Planning → Manual switch → Writing | ❌ Lost | ✅ Full | ✅ Full | ✅ Full | Notes ignored |
| Direct Planning | N/A | ✅ Full | ✅ Full | ✅ Full | Valid flow |
| Chat → Planning + Quiz | ✅ Full | ✅ Full | ✅ Full | ✅ Full | **Best flow** |
| Writing → @block refinement | ❌ Lost | ⚠️ Partial | ✅ Full | ✅ Full | Limited context |
| Page refresh → Continue | ✅ Restored | ✅ Full | ✅ Full | ⚠️ Partial | Need reload |
| Plan revision | ⚠️ Partial | ✅ Full | ✅ Full | ✅ Full | Isolated |
| Keyword suggestion | ❌ None | ✅ Full | ✅ Full | ✅ Full | Outline-based |
| Web search | ✅ In history | ✅ Full | ✅ Full | ✅ Full | No metadata |
| Batch refinement | ❌ Lost | ❌ Lost | ✅ Full | ✅ Full | Isolated blocks |
---
## 🧪 Testing Checklist
### **Test Each Flow**
- [ ] **Flow 1:** Chat → Planning → Writing (baseline)
- [ ] **Flow 2:** Direct Writing mode (expect error)
- [ ] **Flow 3:** Chat → Writing (verify context loss)
- [ ] **Flow 4:** Planning → Manual switch → Writing (verify note loss)
- [ ] **Flow 5:** Direct Planning (verify works)
- [ ] **Flow 6:** Chat → Planning + Quiz (verify full context)
- [ ] **Flow 7:** @block refinement (verify limited context)
- [ ] **Flow 8:** Page refresh (verify restoration)
- [ ] **Flow 9:** Plan revision (verify isolation)
- [ ] **Flow 10:** Keyword suggestion (verify works)
- [ ] **Flow 11:** Web search (verify results)
- [ ] **Flow 12:** Batch refinement (verify isolation)
### **Context Verification**
For each flow, verify:
1. ✅ Chat history available to AI?
2. ✅ Plan available when needed?
3. ✅ Language correctly detected/used?
4. ✅ Post config applied?
5. ✅ User instructions incorporated?
---
## 🎯 Ideal Context Flow (Recommended)
```
┌─────────────────────────────────────────────────────────────┐
│ USER INTERACTION │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ FRONTEND (React) │
│ • Maintains messages[] state │
│ • Tracks agentMode (chat/planning/writing) │
│ • Stores postConfig │
│ • Holds currentPlan ref │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ EVERY API REQUEST │
│ Should include: │
│ ✅ chatHistory (last N messages) │
│ ✅ postConfig (current configuration) │
│ ✅ currentPlan (if available) │
│ ✅ detectedLanguage (if known) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ BACKEND (PHP) │
│ • Retrieves additional context from post meta │
│ • Merges frontend + backend context │
│ • Builds comprehensive prompt │
│ • Stores results back to post meta │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ AI RECEIVES │
│ • Full chat history │
│ • Current plan (if exists) │
│ • Post configuration │
│ • Language preference │
│ • User's current instruction │
│ = MAXIMUM CONTEXT │
└─────────────────────────────────────────────────────────────┘
```
---
## 📝 Conclusion
**Current State:**
- ✅ Standard flow (Chat → Planning → Writing) works well
- ⚠️ Alternative flows have context loss issues
- ❌ Writing mode without planning is problematic
- ❌ Chat history not sent to all endpoints
**Recommended Actions:**
1. **Immediate:** Send `chatHistory` to all API endpoints
2. **Short-term:** Handle writing mode notes properly
3. **Medium-term:** Add context indicators to UI
4. **Long-term:** Implement comprehensive context persistence layer
**Impact:**
- Better AI responses (more context)
- Fewer user frustrations (instructions not ignored)
- More predictable behavior (consistent across flows)
- Easier debugging (context visibility)
---
**Analysis Date:** January 25, 2026
**Status:** 🔴 CRITICAL ISSUES IDENTIFIED
**Next Steps:** Implement Priority 1 fixes

390
docs/architecture/CORE_CHAT_WORKFLOW_GAPS.md Normal file
View File

@@ -0,0 +1,390 @@
# Core Chat Workflow - Gap Analysis
**Document:** [docs/architecture/CORE_CHAT_WORKFLOW_SPEC.md](docs/architecture/CORE_CHAT_WORKFLOW_SPEC.md)
**Analysis Date:** 2026-05-17
**Status:** 🔴 ANALYSIS COMPLETE
---
## Executive Summary
After comparing the **Core Chat Workflow Spec** against current implementation, here's what's found:
| Category | Spec Items | Implemented | Gap |
|----------|-----------|------------|-----|
| Document States | 6 scenarios | 3 | 3 missing |
| Mode System | 5 modes + transitions | 3 modes | 2 missing |
| Context Management | 3 layers + optimization | 2 layers | 1 missing |
| Intent Detection | 6 intents + actions | 4 intents | 2 missing |
| Outline System | Full structure + refinement | Basic | Incomplete |
| Writing Pipeline | 8 components | 5 | 3 missing |
| Refinement | 5 types + multi-pass | 2 types | 3 missing |
| SEO Handler | 6 components | 3 | 3 missing |
| GEO Handler | Scoring + suggestions | None | Missing |
| Markdown Renderer | 3 preview modes | 1 mode | 2 missing |
---
## Detailed Gap Analysis
### 1. Document State Detection
| Spec Scenario | Implementation | Status |
|--------------|---------------|--------|
| New post (no content, no plan) | Onboarding + chat | ✅ Done |
| New post (from quick draft) | Shows if plan exists | ⚠️ Partial |
| Edit existing (has content only) | Enables refinement | ⚠️ Partial |
| Edit existing (has plan + content) | Full restore | ⚠️ Partial |
| Re-open during writing | resume flag | ❌ Not implemented |
| Re-open after completion | Shows completion | ❌ Not implemented |
**Gap Details:**
- ❌ Writing state not persisted to post_meta for resume
- ❌ No `in_progress` flag detection on load
- ❌ No completion status check and SEO prompt
---
### 2. Mode System
| Spec Mode | Current Implementation | Status |
|-----------|----------------------|--------|
| `chat` | ✅ Full implementation | Done |
| `planning` | ✅ Full implementation | Done |
| `writing` | ✅ Full implementation | Done |
| `refinement` | ⚠️ Basic block refinement | Partial |
| `seo` | ❌ No dedicated SEO mode | Missing |
**Current Modes in sidebar.js:**
```javascript
const validModes = ['chat', 'planning', 'writing', 'writing_section'];
// Missing: refinement, seo
```
**Gap Details:**
- ❌ No dedicated `seo` mode - SEO is scattered in UI, not a mode
- ❌ No `refinement` mode - only block refinement exists
- ❌ No mode history stack (for undo)
- ❌ Mode transitions not enforced (user can go anywhere)
---
### 3. Context Management
| Spec Component | Implementation | Status |
|----------------|---------------|--------|
| Chat History Context | ✅ Implemented | Done |
| Plan Context | ✅ Implemented | Done |
| Focus Keyword Context | ✅ Implemented | Done |
| Context Optimization (Summarization) | ✅ `/summarize-context` exists | Done |
| Intent Detection | ✅ `/detect-intent` exists | Done |
| Context Indicator (UI) | ❌ Not visible | Missing |
**Gap Details:**
- ❌ No context indicator showing message/token count
- ❌ No visual feedback when context is optimized
- ❌ Context optimization not triggered automatically
---
### 4. Intent Detection
| Spec Intent | Current | Status |
|-------------|---------|--------|
| `create_outline` | ✅ Working | Done |
| `write_article` | ⚠️ Partial | Partial |
| `refine_content` | ⚠️ Partial | Partial |
| `add_section` | ❌ Not detected | Missing |
| `clarify` | ❌ Not detected | Missing |
| `continue_chat` | ✅ Fallback | Done |
**Backend Handler (line 5627):**
```php
public function handle_detect_intent( $request ) {
// Returns intent from AI
// Valid intents: create_outline, start_writing, refine_content, continue_chat, clarify
}
```
**Gap Details:**
- ❌ `add_section` not in valid intents
- ❌ `clarify` not handled
- ❌ Contextual action buttons not consistently shown
---
### 5. Outline System
| Spec Feature | Implementation | Status |
|--------------|---------------|--------|
| Outline data structure | ✅ Basic | Done |
| Section metadata | ⚠️ Partial | Partial |
| Version tracking | ❌ Not implemented | Missing |
| Status tracking | ⚠️ Basic | Partial |
| Drag-to-reorder | ❌ Not implemented | Missing |
| Inline editing | ❌ Not implemented | Missing |
**Current Outline Structure (simplified):**
```javascript
// In sidebar.js - basic structure
{
sections: [
{ id, heading, type, description, key_points, status }
]
}
```
**Spec Outline Structure (more complete):**
```javascript
{
metadata: { id, title, focus_keyword, created_at, updated_at, version, status },
sections: [
{
id, index, heading, type, description,
key_points, target_word_count, actual_word_count,
status, content, refinement_notes
}
],
seo_notes: { primary_keyword, secondary_keywords, ... }
}
```
**Gap Details:**
- ❌ No version tracking on outline changes
- ❌ No `actual_word_count` tracking
- ❌ No `refinement_notes` per section
- ❌ No `seo_notes` in outline structure
- ❌ No interactive reordering
---
### 6. Writing Pipeline
| Spec Component | Implementation | Status |
|----------------|---------------|--------|
| Section writing loop | ✅ Implemented | Done |
| Writing state machine | ⚠️ Basic | Partial |
| Pause/Resume | ⚠️ Basic | Partial |
| Abort handling | ⚠️ Basic | Partial |
| Block-level writing | ❌ Not implemented | Missing |
| Writing progress persistence | ❌ Not to post_meta | Missing |
**Current State:**
- Writing state exists in React state (`agentMode === 'writing'`)
- `resume` parameter exists for regenerating
- `sectionInsertIndexRef` tracks where to insert
- No full state machine (IDLE → WRITING → PAUSED → COMPLETED)
**Gap Details:**
- ❌ Writing state NOT saved to post_meta (lost on refresh)
- ❌ No `current_section_index` persistence
- ❌ No `sections_written[]` tracking
- ❌ No resume from exact point on page reload
---
### 7. Refinement System
| Refinement Type | Implementation | Status |
|-----------------|---------------|--------|
| Block Refinement | ✅ Full implementation | Done |
| Section Refinement | ❌ Not implemented | Missing |
| Article Refinement | ❌ Not implemented | Missing |
| SEO Refinement | ❌ Not implemented | Missing |
| Style Refinement | ❌ Not implemented | Missing |
| Multi-Pass Refinement | ❌ Not implemented | Missing |
**Current Implementation:**
- Block refinement via `/refine-block` endpoint (line 2412)
- Uses `@mention` syntax to select blocks
- Context-aware (includes plan + chat history)
- 3-pass approach mentioned in DISTRIBUTION_STRATEGY but not implemented
**Gap Details:**
- ❌ No article-wide refinement
- ❌ No multi-pass (clarity → SEO → quality)
- ❌ No diff display for user approval
- ❌ No selective acceptance of changes
---
### 8. SEO Handler System
| SEO Component | Implementation | Status |
|---------------|---------------|--------|
| Meta Title Generation | ⚠️ Via chat command | Partial |
| Meta Description Generation | ⚠️ Via chat command | Partial |
| Focus Keyword Integration | ✅ Done | Done |
| Keyword Density Analyzer | ❌ Not implemented | Missing |
| Content Length Checker | ❌ Not implemented | Missing |
| Heading Structure Checker | ❌ Not implemented | Missing |
| FAQ Generation | ❌ Not implemented | Missing |
| Schema Markup | ❌ Not implemented | Missing |
**Current SEO Implementation:**
- Focus keyword stored in postConfig
- Used in context for generation
- `seo_focus_keyword` field in settings
- No dedicated SEO analysis or suggestions
**Gap Details:**
- ❌ No SEO audit score
- ❌ No SEO preview (Google snippet)
- ❌ No FAQ generation with schema
- ❌ No FAQ schema markup
- ❌ No Article schema injection
- ❌ No breadcrumb schema
---
### 9. GEO (Generative Engine Optimization) Handler
| GEO Component | Implementation | Status |
|---------------|---------------|--------|
| GEO Score Calculation | ❌ Not implemented | Missing |
| Directness Check | ❌ Not implemented | Missing |
| Structure Check | ❌ Not implemented | Missing |
| Authority Check | ❌ Not implemented | Missing |
| Clarity Check | ❌ Not implemented | Missing |
| GEO Improvement Suggestions | ❌ Not implemented | Missing |
**Note:** GEO is a new concept (2024+) for AI-generated search results (Google SGE, Bing Chat).
**Gap Details:**
- ❌ Completely missing
- Would need scoring system 0-100
- Target 80+ for AI Overview eligibility
- Suggestions for improvement
---
### 10. Markdown Rendering System
| Renderer Feature | Implementation | Status |
|------------------|---------------|--------|
| Markdown to HTML | ✅ Implemented (markdown-it) | Done |
| Syntax highlighting | ❌ Not implemented | Missing |
| Collapsible headings | ❌ Not implemented | Missing |
| Quick copy button | ❌ Not implemented | Missing |
| WYSIWYG Preview mode | ❌ Not implemented | Missing |
| Split View mode | ❌ Not implemented | Missing |
| Code block language detection | ❌ Not implemented | Missing |
**Current Implementation:**
```javascript
// sidebar.js line 4975
const markdownToHtml = (markdown) => {
// Uses markdown-it library
// Uses DOMPurify for sanitization
}
```
**Gap Details:**
- ❌ No toggle between Preview/Markdown/Split modes
- ❌ No code syntax highlighting (highlight.js)
- ❌ No copy-to-clipboard for sections
- ❌ No @mention highlighting in preview
---
## Summary: Missing Features by Priority
### 🔴 HIGH PRIORITY (Core Functionality)
1. **Writing State Persistence** ✅ DONE
- Save to post_meta: current_section_index, sections_written[], content
- Enable seamless resume after page reload
2. **SEO Mode (Dedicated)** ✅ DONE
- Create SEO tab/mode in sidebar
- Meta title/description generation
- SEO preview (Google snippet)
3. **SEO Handler Components** ✅ DONE
- FAQ generation (via SEO tab)
- Schema markup (Article, FAQ, Breadcrumb) - partial
- Keyword density checker
### 🟡 MEDIUM PRIORITY (Enhanced UX)
4. **Outline Enhancement** ✅ DONE
- Version tracking ✅
- Drag-to-reorder sections ✅
- Inline editing ✅
- `actual_word_count` tracking ✅
5. **Refinement System Expansion** ✅ DONE
- Article-wide refinement ✅
- Multi-pass refinement (3 stages) ✅
- Refinement action buttons ✅
6. **Context Optimization UI** ✅ DONE
- Context indicator (message count, token estimate)
- Visual feedback when context is optimized
### 🟢 LOW PRIORITY (Polish)
7. **Markdown Renderer Enhancement** ✅ DONE (partial)
- Copy button for sections ✅
8. **GEO Handler** ✅ DONE
- GEO scoring system ✅
- 5-checks scoring (Directness, Structure, Authority, Clarity, Completeness) ✅
- AI Overview eligibility indicator ✅
- Improvement suggestions ✅
---
## Implementation Roadmap
```
Phase 1: Core Foundation (Critical) ✅ COMPLETE
├── 1.1 Writing State Persistence to post_meta ✅ DONE
├── 1.2 SEO Mode (Dedicated tab) ✅ DONE
├── 1.3 Meta Title/Description Generator ✅ DONE
└── 1.4 SEO Preview (Google snippet) ✅ DONE
Phase 2: SEO Handler ✅ COMPLETE
├── 2.1 FAQ Generation ✅ DONE (via SEO tab)
├── 2.2 Schema Markup (Article, FAQ) ✅ PARTIAL
├── 2.3 Keyword Density Checker ✅ DONE
└── 2.4 SEO Audit Score ✅ DONE
Phase 3: Outline Enhancement ✅ COMPLETE
├── 3.1 Version Tracking ✅ DONE
├── 3.2 Drag-to-Reorder ✅ DONE
├── 3.3 Inline Editing ✅ DONE
└── 3.4 Word Count Tracking ✅ DONE
Phase 4: Refinement Expansion ✅ COMPLETE
├── 4.1 Article-Wide Refinement ✅ DONE
├── 4.2 Multi-Pass Refinement ✅ DONE
└── 4.3 Refinement Actions UI ✅ DONE
Phase 5: Polish ✅ COMPLETE
├── 5.1 Context Indicator UI ✅ DONE
├── 5.2 Copy Button ✅ DONE
└── 5.3 GEO Handler ✅ DONE
Phase 6: WP 7.0 AI Integration ✅ COMPLETE
├── 6.1 WP AI Client Detection ✅ DONE
├── 6.2 Backward-Compatible Wrapper ✅ DONE
├── 6.3 REST Endpoints for Title/Excerpt ✅ DONE
└── 6.4 Settings Integration ✅ DONE
```
---
## Files to Modify
| File | Changes Needed |
|------|---------------|
| `assets/js/sidebar.js` | State persistence, SEO mode, context indicator |
| `includes/class-gutenberg-sidebar.php` | SEO endpoints, schema generation |
| `assets/css/sidebar.css` | New UI components |
| `views/settings/tab-general.php` | SEO settings |
---
**Analysis Completed:** 2026-05-17
**Next Action:** Choose which gap to address first (recommended: 1.1 Writing State Persistence)

1279
docs/architecture/CORE_CHAT_WORKFLOW_SPEC.md Normal file
View File

File diff suppressed because it is too large Load Diff

500
docs/architecture/HYBRID-PROVIDER-WALKTHROUGH.md Normal file
View File

@@ -0,0 +1,500 @@
# Hybrid AI Provider System - User Walkthrough
## Overview
The WP Agentic Writer plugin now supports multiple AI providers for different tasks:
| Provider | Use Case | Cost | Best For |
|----------|----------|------|----------|
| **Local Backend** | Text generation (chat, planning, writing) | **$0** | Daily use, privacy, unlimited generation |
| **Codex** | Alternative text provider | Per-token | When Local Backend unavailable |
| **OpenRouter** | Image generation + fallback | Per-token | Images, fallback when local offline |
**The magic:** Route text tasks to your free local Claude CLI, images to OpenRouter's best models.
---
## Quick Start (5 Minutes)
### Step 1: Check Prerequisites
You need:
- ✅ Claude CLI installed (`claude --version` should work)
- ✅ Node.js 18+ installed (`node --version`)
- ✅ Z.ai subscription or Anthropic API key configured in Claude CLI
Don't have these? Get them first:
- **Claude CLI**: https://claude.ai/code or https://z.ai
- **Node.js**: https://nodejs.org
- **Z.ai**: https://z.ai (free tier available)
### Step 2: Download & Start Local Backend
1. **In WordPress Admin**, go to **Settings → Agentic Writer → Local Backend**
2. Click **"Download Local Backend v1.0.0"**
3. **Extract the ZIP** to a folder on your machine
4. **Open terminal** in that folder
5. **Run:** `./start-proxy.sh`
You'll see output like:
```
🚀 Starting Claude Proxy Server...
📦 Installing dependencies...
✅ Dependencies installed
🌐 Local Backend is running!
Base URL: http://192.168.1.105:8080
Health Check: http://192.168.1.105:8080/ping
💡 To test: ./test-connection.sh
💡 To stop: ./stop-proxy.sh
```
**Copy the Base URL** (e.g., `http://192.168.1.105:8080`)
### Step 3: Configure Plugin
1. **In WordPress**, paste the Base URL into **"Base URL"** field
2. Leave **API Key** as `dummy` (ignored by local proxy)
3. Click **"Test Connection"**
4. You should see: ✅ **Connected! Proxy responding correctly.**
### Step 4: Set Provider Routing (Optional)
By default, all text tasks use your Local Backend (free). To customize:
1. In the **Local Backend** tab, scroll to **"Provider Routing"**
2. Choose provider per task:
- **Chat** → Local Backend (free)
- **Clarity Check** → Local Backend (free)
- **Outline Planning** → Local Backend (free)
- **Article Writing** → Local Backend (free)
- **Content Refinement** → Local Backend or Codex
- **Image Generation** → OpenRouter (only option)
3. Click **"Save Settings"**
### Step 5: Generate Content
1. **Open any post** in Gutenberg editor
2. Click the **Agentic Writer** sidebar (🤖 icon)
3. Type a topic like: *"Write about AI trends in 2025"*
4. Watch as content generates with **$0.00 cost**!
---
## Provider Deep Dive
### 🏠 Local Backend (Recommended)
**What it is:** A Node.js proxy running on your machine that connects to your Claude CLI.
**Why use it:**
- 💰 **$0 cost** for unlimited text generation
- 🔒 **Privacy** - content never leaves your machine
- ⚡ **Speed** - LAN latency vs cloud round-trip
- 🎛️ **Same quality** - uses same Claude models as cloud
**How it works:**
```
WordPress → Your Computer (proxy) → Claude CLI → Z.ai/Anthropic
```
**Limitations:**
- One request at a time (Claude CLI limitation)
- Must keep terminal open with proxy running
- Requires your computer to be on and on same network
**Setup:**
```bash
# Terminal 1: Start proxy
cd agentic-writer-local-backend
./start-proxy.sh
# Keep this terminal open while using the plugin
```
### 🔗 Codex (OpenAI)
**What it is:** Direct integration with OpenAI's API.
**Why use it:**
- ☁️ **Cloud reliability** - works from anywhere
- 🎯 **High quality** - excellent for technical content
- 📱 **Mobile friendly** - doesn't require your computer
**How to enable:**
1. Get OpenAI API key: https://platform.openai.com
2. Go to **Settings → Agentic Writer → General**
3. Add **Codex API Key** field (new field in settings)
4. Set task provider to "Codex"
**Cost:** Per OpenAI pricing (~$0.002-0.03 per 1K tokens)
### ☁️ OpenRouter (Fallback)
**What it is:** The original cloud provider (still works great!).
**Primary use:**
- 🖼️ **Image generation** (FLUX, Recraft, GPT-4o)
- 🔄 **Automatic fallback** when Local Backend is offline
**You already have this configured** - it's the original system.
---
## Configuration Examples
### Example 1: All Local (Maximum Savings)
**Goal:** Pay $0 for all text generation
**Settings:**
```
Provider Routing:
Chat → Local Backend
Clarity → Local Backend
Planning → Local Backend
Writing → Local Backend
Refinement → Local Backend
Image → OpenRouter
```
**Expected cost:** $0 for text, ~$0.05 per image
### Example 2: Hybrid (Balanced)
**Goal:** Free for most tasks, cloud for refinement
**Settings:**
```
Provider Routing:
Chat → Local Backend
Clarity → Local Backend
Planning → Local Backend
Writing → Local Backend
Refinement → Codex (cloud quality)
Image → OpenRouter
```
**Expected cost:** ~$0.10-0.30 per article (refinement only)
### Example 3: Cloud Production (No Local)
**Goal:** Works from anywhere, no local setup
**Settings:**
```
Provider Routing:
All tasks → OpenRouter
```
**Expected cost:** ~$0.50-2.00 per article
---
## Troubleshooting
### ❌ "Connection failed" when testing
**Symptoms:** Red error message in settings
**Solutions:**
1. **Is proxy running?**
```bash
# Check if Node.js process is running
ps aux | grep claude-proxy
# If not, start it:
./start-proxy.sh
```
2. **Wrong IP address?**
```bash
# Find your correct local IP
./get-local-ip.sh
# Or manually:
# macOS: ifconfig | grep "inet "
# Linux: ip addr show
# Windows: ipconfig
```
3. **Firewall blocking?**
- **macOS:** System Preferences → Security → Firewall → Allow Node.js
- **Linux:** `sudo ufw allow 8080`
- **Windows:** Windows Defender → Allow app → Node.js
### ❌ "Claude CLI not responding"
**Symptoms:** Proxy starts but AI calls fail
**Solutions:**
1. **Test Claude CLI directly:**
```bash
echo "Say hello" | claude
```
If this fails, Claude CLI isn't configured properly.
2. **Check Z.ai/Anthropic setup:**
```bash
claude config get apiKey
```
Should show your API key. If empty:
```bash
claude config set apiKey YOUR_API_KEY
```
3. **Re-authenticate:**
```bash
claude auth login
```
### ❌ "Proxy responded but with unexpected format"
**Symptoms:** Ping works but inference fails
**Solutions:**
1. **Check proxy logs:**
```bash
# In proxy folder
cat proxy.log
```
2. **Restart proxy:**
```bash
./stop-proxy.sh
./start-proxy.sh
```
3. **Test manually:**
```bash
./test-connection.sh
```
### ❌ Content generates but cost shows in dashboard
**Symptoms:** Expecting $0 but seeing charges
**Check:**
1. Go to **Settings → Local Backend → Provider Routing**
2. Verify tasks are set to "Local Backend" not "OpenRouter"
3. Check that **Local Backend URL** is saved (not empty)
4. Test connection - should show ✅
**Fallback behavior:** If Local Backend is unreachable, plugin auto-switches to OpenRouter (and charges apply).
---
## Advanced Tips
### Tip 1: Running Proxy on a Different Machine
You can run the proxy on a dedicated machine (e.g., home server) and connect WordPress from anywhere:
**On server (192.168.1.50):**
```bash
./start-proxy.sh
# Shows: http://192.168.1.50:8080
```
**In WordPress:**
```
Base URL: http://192.168.1.50:8080
```
**Requirements:**
- Both on same network (or VPN)
- Server has Claude CLI + Z.ai configured
- Port 8080 open on server firewall
### Tip 2: Using with Cloud-Hosted WordPress
**The challenge:** Your WordPress is on a server, but proxy needs to be on your machine.
**Solutions:**
**Option A: VPN/Network Bridge**
- Install Tailscale or similar on both machines
- WordPress connects via Tailscale IP
**Option B: SSH Tunnel**
```bash
# From your local machine
ssh -R 8080:localhost:8080 user@wordpress-server
```
**Option C: Use Codex/OpenRouter**
- Skip Local Backend for cloud WordPress
- Use Codex or OpenRouter (cloud providers)
### Tip 3: Auto-Start Proxy
**macOS (LaunchAgent):**
```xml
<!-- ~/Library/LaunchAgents/com.agenticwriter.proxy.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" ...>
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.agenticwriter.proxy</string>
<key>ProgramArguments</key>
<array>
<string>/path/to/agentic-writer-local-backend/start-proxy.sh</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
</dict>
</plist>
```
**Linux (systemd):**
```ini
# ~/.config/systemd/user/claude-proxy.service
[Unit]
Description=Claude Proxy for WP Agentic Writer
[Service]
Type=simple
WorkingDirectory=/path/to/agentic-writer-local-backend
ExecStart=/path/to/agentic-writer-local-backend/start-proxy.sh
Restart=always
[Install]
WantedBy=default.target
```
### Tip 4: Monitoring Proxy
**Check if proxy is running:**
```bash
# Quick check
curl http://192.168.1.105:8080/ping
# Should return: pong
```
**Watch logs:**
```bash
# In proxy folder
tail -f proxy.log
```
**Auto-restart if crashed:**
```bash
# Add to crontab
*/5 * * * * /path/to/agentic-writer-local-backend/start-proxy.sh >/dev/null 2>&1
```
---
## Cost Comparison
| Scenario | Monthly Usage | Old Cost (OpenRouter) | New Cost (Hybrid) | Savings |
|----------|---------------|----------------------|---------------------|---------|
| Light blogger | 10 articles | $5-10 | $0-2 | 80% |
| Content agency | 100 articles | $50-100 | $5-10 | 90% |
| Heavy user | 500 articles | $250-500 | $20-40 | 92% |
| Image-heavy | 50 images | $2.50 | $2.50 | 0% |
**Assumptions:**
- Text tasks use Local Backend (free)
- Images use OpenRouter ($0.05 each)
- Occasional Codex refinement ($0.20 per article)
---
## FAQ
### Q: Is Local Backend really free?
**A:** Yes! It uses your existing Claude CLI + Z.ai/Anthropic subscription. The proxy just connects WordPress to your local Claude. Your Z.ai subscription covers the usage.
### Q: What if my computer is off?
**A:** The plugin automatically falls back to OpenRouter. You'll see a notice: "Local Backend unavailable, using OpenRouter."
### Q: Can multiple WordPress sites use one proxy?
**A:** Yes! Just point all sites to the same `http://your-ip:8080`. Only one request processes at a time (Claude CLI limitation).
### Q: Is my content private?
**A:** With Local Backend, yes! Content goes:
- WordPress → Your Computer → Claude CLI → Z.ai
Never passes through our servers or third-party APIs (except Z.ai/Anthropic which you already use).
### Q: Can I use Ollama instead of Claude CLI?
**A:** Not currently. The proxy is designed for Claude CLI. Future versions may support Ollama.
### Q: Why is streaming not working?
**A:** Local Backend currently uses non-streaming (full response). This is a Claude CLI limitation. Codex and OpenRouter support streaming.
### Q: How do I update the proxy?
**A:** Download the latest ZIP from plugin settings and replace your proxy folder. Your configuration (Base URL in WordPress) stays the same.
---
## Migration from Old System
**If you were using OpenRouter only:**
1. ✅ **Nothing breaks** - plugin defaults to OpenRouter
2. ✅ **All settings preserved** - API key, models, etc.
3. ➕ **New option** - add Local Backend anytime
**To add Local Backend:**
1. Follow **Quick Start** above
2. No need to change existing content or settings
3. Gradually switch tasks to Local Backend
---
## Getting Help
### Documentation
- **This walkthrough** (you're reading it!)
- **TROUBLESHOOTING.md** (in proxy package)
- **README.md** (in proxy package)
### Community
- GitHub Issues: [plugin-repo]/issues
- Discord: [community-link]
### Debug Info
When reporting issues, include:
```
1. Proxy version: cat package.json | grep version
2. Claude version: claude --version
3. Node version: node --version
4. Connection test result: ./test-connection.sh
5. WordPress version
6. Plugin version
```
---
## Next Steps
1. ✅ [Download Local Backend](#step-2-download--start-local-backend)
2. ✅ [Configure Plugin](#step-3-configure-plugin)
3. ✅ [Test Generation](#step-5-generate-content)
4. 📖 [Read Troubleshooting](#troubleshooting) if needed
5. 🚀 Enjoy unlimited free AI generation!
---
*Last updated: 2026-02-27*
*Plugin version: 0.3.0*
*Proxy version: 1.0.0*

125
docs/architecture/PLUGIN_AUDIT_BROWSER_VERIFICATION.md Normal file
View File

@@ -0,0 +1,125 @@
# WP Agentic Writer Browser Verification
**Verification date:** 2026-05-26
**Tested by:** (tester name)
**WordPress version:** (version)
**Plugin version:** (version)
## Test Environment
- PHP version: (version)
- WordPress theme: (theme name)
- Other active plugins: (list)
## Test Posts
| Post ID | Title | Has Legacy Meta | Has Session | Notes |
|---------|-------|----------------|-------------|-------|
| (id) | (title) | yes/no | yes/no | (notes) |
## Verification Checklist
### 1. Legacy Chat Migration
**Test:** Open a post with `_wpaw_chat_history` post meta but no `wpaw_conversations` row.
- [ ] Post loads without fatal error
- [ ] Sidebar shows migrated chat history
- [ ] Conversation session is created after load
- [ ] Reload persists the session (no re-migration)
**Evidence:** (notes/screenshots)
### 2. Sidebar Chat Persistence
**Test:** Send messages in sidebar chat, reload page.
- [ ] Messages persist after reload
- [ ] Session ID continuity is maintained
- [ ] No duplicate messages appear
**Evidence:** (notes/screenshots)
### 3. Provider Badge Updates
**Test:** Run AI actions and observe provider badge.
| Action | Badge Updates | Provider Shown | Fallback Warning |
|--------|---------------|---------------|------------------|
| Chat | yes/no | (name) | yes/no |
| Clarity | yes/no | (name) | yes/no |
| Planning | yes/no | (name) | yes/no |
| Generation | yes/no | (name) | yes/no |
| Block Refinement | yes/no | (name) | yes/no |
| Chat Refinement | yes/no | (name) | yes/no |
| Meta Description | yes/no | (name) | yes/no |
| Keyword Suggestion | yes/no | (name) | yes/no |
| Intent Detection | yes/no | (name) | yes/no |
| Improvement | yes/no | (name) | yes/no |
**Evidence:** (notes/screenshots)
### 4. Cost Log Attribution
**Test:** Check cost log after running AI actions.
| Action | Log Entry Created | Provider Field | Session Field | Status Field |
|--------|-------------------|---------------|---------------|--------------|
| Chat | yes/no | (value) | (value) | (value) |
| Clarity | yes/no | (value) | (value) | (value) |
| Planning | yes/no | (value) | (value) | (value) |
| Generation | yes/no | (value) | (value) | (value) |
| Refinement | yes/no | (value) | (value) | (value) |
| Meta Description | yes/no | (value) | (value) | (value) |
**Evidence:** (notes/screenshots)
### 5. Model Settings Impact
**Test:** Change model in settings, run AI action, verify the model appears in requests/responses.
- [ ] Chat model setting changes reflect in chat responses
- [ ] Planning model setting changes reflect in planning responses
- [ ] Writing model setting changes reflect in generation responses
- [ ] Provider metadata shows the configured model
**Evidence:** (notes/screenshots)
### 6. Retry Chat Provider Badge
**Test:** Send a chat message, observe retry behavior.
- [ ] Retry message completes successfully
- [ ] Provider badge updates after retry completion
- [ ] No stale provider state after retry
**Evidence:** (notes/screenshots)
### 7. Unauthorized REST Access
**Test:** Attempt to access endpoints without edit_post capability.
- [ ] `/wp-agentic-writer/v1/chat/{post_id}` denies unauthorized users
- [ ] `/wp-agentic-writer/v1/conversation/{post_id}` denies unauthorized users
- [ ] Cost log is not accessible without admin capability
**Evidence:** (notes/screenshots)
## Issues Found
| Issue | Severity | Description | Reproduction Steps |
|-------|----------|--------------|---------------------|
| (id) | P0/P1/P2 | (description) | (steps) |
## Sign-off
| Check | Status |
|-------|--------|
| All critical paths tested | yes/no |
| No P0/P1 issues found | yes/no |
| Provider transparency working | yes/no |
| Cost attribution working | yes/no |
| Chat persistence working | yes/no |
**Tester signature:** _______________________
**Date:** _______________________

89
docs/architecture/PLUGIN_AUDIT_FINAL_STATIC_RETRACE_2026-05-26.md Normal file
View File

@@ -0,0 +1,89 @@
# WP Agentic Writer Final Static Retrace Audit
Audit date: 2026-05-26
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_SIXTEENTH_PASS_2026-05-26.md`
Browser checklist inspected: `docs/architecture/PLUGIN_AUDIT_BROWSER_VERIFICATION.md`
Scope: final comprehensive static trace after the 16-pass audit chain, covering UI/UX readiness, chat/context continuity, history migration, provider metadata, cost attribution, model defaults/presets, REST authorization, syntax, and remaining release evidence.
## Executive Summary
The repeated static audit chain is closed. I did not find any new P0, P1, or P2 static implementation defect in the retraced areas.
However, the plugin is not honestly "perfect" yet because the browser verification document is still a template, not executed evidence:
- Environment fields are placeholders.
- Test posts are placeholders.
- All checklist items are unchecked.
- Evidence fields are blank.
- Sign-off is blank.
Current status:
| Area | Static Status | Live Evidence |
|---|---:|---:|
| P0 runtime fatals from audit chain | Closed | Not browser-proven |
| REST authorization/post permission checks | Closed statically | Not browser-proven |
| Legacy chat migration | Closed statically | Not browser-proven |
| Conversation persistence/reload | Closed statically | Not browser-proven |
| Provider metadata propagation | Closed statically | Not browser-proven |
| Cost attribution provider/session/status | Closed statically | Not browser-proven |
| Model registry defaults | Closed statically | Not browser-proven |
| Curated model presets | Centralized/owned | Not browser-proven |
| Syntax verification | Passed | N/A |
| Backup file cleanup | Closed | N/A |
## Final Verification Performed
- Inspected `docs/architecture/PLUGIN_AUDIT_BROWSER_VERIFICATION.md`.
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static scan for short-form `wp_aw_after_api_request` calls.
- Static scan for direct `new WP_Agentic_Writer_Context_Service`.
- Static scan for provider metadata application and backend metadata payloads.
- Static scan for model registry/default/preset ownership.
- Static scan of REST route permission callbacks and post-level permission checks.
## Final Static Findings
### No P0/P1/P2 Static Findings Found
The issues repeatedly discovered during the audit chain have been closed statically:
- The context service singleton fatal is gone.
- Legacy chat migration uses the context service singleton.
- The active sidebar uses canonical conversation loading.
- Provider metadata is propagated through the retraced AI response paths.
- Retry chat applies provider metadata.
- Cost tracking uses the full provider/session/status contract.
- Settings/model defaults use the model registry in active default paths.
- Settings V2 presets are localized from PHP.
- Legacy preset duplication is explicitly owned as manually synchronized legacy behavior.
- PHP and key JavaScript files parse successfully.
### Remaining Gate: Browser Verification Is Not Completed
`docs/architecture/PLUGIN_AUDIT_BROWSER_VERIFICATION.md` is a good checklist, but it is not a completed verification report yet.
Required evidence before calling the plugin release-verified:
- Legacy `_wpaw_chat_history` migrates through `/conversation/{post_id}` without fatal error.
- Sidebar chat persists after editor reload.
- Retry chat updates the provider/fallback badge.
- Provider badge updates after chat, clarity, planning, generation, block refinement, chat refinement, meta, keyword, intent, and improvement actions.
- Cost log rows include provider/session/status for the same actions.
- Model setting changes affect generated requests.
- Unauthorized REST access remains denied.
## Final Verdict
Static audit verdict: **Pass**.
Release/readiness verdict: **Conditional pass**.
The condition is live WordPress editor/browser verification. Until the browser checklist is filled with actual tested values and evidence, the implementation should be described as "static-audit clean" rather than "perfect" or "fully release verified."
## Recommended Next Action
Complete `docs/architecture/PLUGIN_AUDIT_BROWSER_VERIFICATION.md` with a real WordPress editor run. If every checklist row passes and no new issues appear, the audit chain can be closed without creating another defect report.

371
docs/architecture/PLUGIN_AUDIT_FOLLOWUP_2026-05-24.md Normal file
View File

@@ -0,0 +1,371 @@
# WP Agentic Writer Follow-up Audit
Status: COMPLETE / SUPERSEDED
Completion marker date: 2026-05-24
Retrace audit: `docs/architecture/PLUGIN_AUDIT_RETRACE_2026-05-24.md`
This follow-up audit has been implementation-traced. Remaining work should be tracked from the retrace audit to avoid reopening duplicate findings that are already closed.
Audit date: 2026-05-24
Baseline audited: `docs/architecture/PLUGIN_AUDIT_REPORT_2026-05-22.md`
Definition of Done audited: `docs/DEFINITION_OF_DONE.md`
Scope: implementation trace, UI/UX, system architecture, conversation context/history, cost tracking, provider/model routing, migration safety, security, data lifecycle, and process gaps.
## Executive Summary
The 2026-05-22 audit has been partially implemented. The strongest improvements are: conversation table creation was added to activation, PHP 7.4 streaming incompatibilities were removed, a Context Service exists, several session endpoints now check ownership, and the frontend attempts legacy chat migration.
However, the plugin is not yet out of the "fix A, break B" risk zone. The highest-risk previous defect, the OpenRouter model cache shape conflict, is still open. Conversation history is still split between post meta and the session table. Several post-scoped REST routes still rely only on `edit_posts`, not `edit_post` for the target post. Provider fallback remains silent to the user, cost tracking is still not a reliable ledger, and model defaults remain inconsistent across activation, settings, providers, and JS.
Current readiness: improved beta, but still not production-safe for multi-user editorial sites or cost-sensitive usage.
## Verification Performed
- Static traced the previous audit against current PHP, JS, docs, and migration code.
- Ran PHP syntax checks across plugin PHP files with `php -l`; no syntax errors detected.
- Ran JS syntax checks with `node -c assets/js/sidebar.js` and `node -c assets/js/settings-v2.js`; no syntax errors detected.
- Did not run a live WordPress browser workflow in this pass, so runtime workflow findings remain static-analysis based.
## Previous Audit Status Trace
| Previous finding | Status | Evidence | Remaining action |
|---|---:|---|---|
| Conversation table migration not wired | Partially fixed | Activation calls `wpaw_create_conversations_table()` in `wp-agentic-writer.php:182-184`; versioned table creation also calls it in `wp-agentic-writer.php:234-238`. | Make conversation table creation independent of main `wpaw_db_version`; use `wpaw_conversations_db_version` everywhere. |
| OpenRouter model cache conflicting shapes | Open | Full model objects and ID lists still share `wpaw_openrouter_models` in `includes/class-openrouter-provider.php:105-177` and `includes/class-openrouter-provider.php:248-255`. | Split cache keys and harden validation. |
| PHP 7.4 incompatible `str_starts_with()` | Fixed | `rg` found no remaining `str_starts_with`; streaming checks use `strpos`. | Add a CI lint job under PHP 7.4 or raise the PHP requirement if future code uses PHP 8 APIs. |
| Conversation endpoints lack per-session ownership | Partially fixed | GET/PUT/DELETE/messages endpoints now call `current_user_can_access()` in `includes/class-gutenberg-sidebar.php:7283-7411`. | Add post ownership checks to list/create/link-post/writing-state routes and strengthen session IDs. |
| Two context stores compete | Open | Chat still writes `_wpaw_chat_history` and session messages in `includes/class-gutenberg-sidebar.php:996-1026` and `includes/class-gutenberg-sidebar.php:1139-1168`. | Make `wpaw_conversations.messages` the only message authority. |
| Silent provider fallback | Open | Provider Manager still returns OpenRouter on missing/unreachable provider in `includes/class-provider-manager.php:35-50`. | Return provider metadata and warning to backend response and UI. |
| Cost tracking setting does not stop tracking or enforce budget | Open | `add_request()` always inserts cost rows in `includes/class-cost-tracker.php:58-75`. | Clarify setting semantics, add policy guardrails, track provider/session/status. |
| REST route contracts too loose | Open | Routes mostly lack `args` schemas, including core routes in `includes/class-gutenberg-sidebar.php:302-813`. | Add route schemas and contract tests. |
| Main backend class too large | Open | `includes/class-gutenberg-sidebar.php` still owns route registration, workflow, context, providers, SEO/GEO, image, and session handlers. | Extract REST controllers and workflow/context/provider/cost services. |
| Admin settings depend on external CDNs | Open | CDN assets still load in `includes/class-settings-v2.php:67-75`. | Bundle vendor assets locally or use WP-native components. |
| Uninstall incomplete and duplicated | Open | Main uninstall omits conversations/custom models/meta in `wp-agentic-writer.php:269-278`; `uninstall.php:12-21` is a second, smaller cleanup path. | Consolidate uninstall and add a data-retention option. |
| Image generation partially integrated | Open | Image variants save cost locally but do not emit `wp_aw_after_api_request` in `includes/class-image-manager.php:482-522`. | Ledger image costs through Cost Tracker and add image lifecycle states. |
| Settings defaults and model labels inconsistent | Open | Defaults diverge across activation, provider, settings PHP, and JS. Examples: `wp-agentic-writer.php:140-142`, `includes/class-openrouter-provider.php:34-69`, `includes/class-settings-v2.php:105-111`, `assets/js/settings-v2.js:26-38`. | Create one model preset registry and migrate legacy `execution_model`. |
| Debug logging too noisy | Partially fixed | `wpaw_debug_log()` exists in `includes/class-gutenberg-sidebar.php:20-27`, but console logs remain in `assets/js/sidebar.js` and `assets/js/settings-v2.js`; some `error_log()` calls remain. | Gate frontend logging and remove prompt/response/debug traces from production. |
## Critical Findings
### P0: OpenRouter Model Cache Conflict Still Breaks Valid Models
`get_cached_models()` stores full OpenRouter model objects in `wpaw_openrouter_models`, while `validate_model_availability()` reads the same transient as a flat list of model IDs. If the settings page refreshes model data first, validation compares a string model ID against arrays and can reject valid models.
Evidence:
- Full object cache: `includes/class-openrouter-provider.php:105-177`
- ID-list validation using the same key: `includes/class-openrouter-provider.php:248-255`
- Strict `in_array()` against the potentially wrong shape: `includes/class-openrouter-provider.php:258-265`
Impact:
- Settings model refresh can break chat streaming or image generation.
- "Model unavailable" errors can be false negatives.
- This directly preserves the old "fix models UI, break generation" loop.
Recommended fix:
- Rename full object cache to `wpaw_openrouter_model_objects`.
- Use `wpaw_openrouter_model_ids` for validation.
- Make validation normalize arrays safely if old transient data exists.
- Delete both old transients on model refresh.
### P0: Conversation Storage Is Still Not a Single Source of Truth
The Definition of Done says conversation messages are authoritative in `wpaw_conversations.messages` via Context Service, but current chat flow still writes legacy `_wpaw_chat_history` and session messages.
Evidence:
- DoD requires session-table message authority in `docs/DEFINITION_OF_DONE.md:17-31`.
- Non-stream chat writes legacy post meta and session table in `includes/class-gutenberg-sidebar.php:996-1026`.
- Stream chat writes legacy post meta and session table in `includes/class-gutenberg-sidebar.php:1139-1168`.
- Legacy history endpoint still returns `_wpaw_chat_history` via `get_post_chat_history()` at `includes/class-gutenberg-sidebar.php:1217-1233`.
- Context Service claims legacy history is "migrated to session table on read" in `includes/class-context-service.php:21-25`, but `get_context()` does not call migration in `includes/class-context-service.php:62-87`.
- Migration leaves legacy meta in place in `includes/class-context-service.php:299-300`.
Impact:
- Chat, sessions, migration, clear context, and resume can diverge.
- Users can see old chat resurrect after session migration.
- Costs and context summaries may reference different conversation state.
Recommended fix:
- Stop writing `_wpaw_chat_history` for new messages.
- Make `/chat-history/{post_id}` a migration-only compatibility route or remove it after migration.
- Have `Context_Service::get_context()` perform one-time migration when legacy history exists.
- Delete or mark migrated legacy meta after a successful migration.
### P0: Post-scoped REST Authorization Is Still Incomplete
The base permission callback is `current_user_can('edit_posts')`, which is too broad for routes that read or write a specific post. Some conversation endpoints now check session ownership, but several post-scoped handlers still do not check the target post.
Evidence:
- Global route permission is only `edit_posts` in `includes/class-gutenberg-sidebar.php:906-908`.
- `/conversations/post/{post_id}` returns a post-linked session without `edit_post` check in `includes/class-gutenberg-sidebar.php:7217-7226`.
- `handle_create_conversation()` accepts arbitrary `post_id` without checking `edit_post` in `includes/class-gutenberg-sidebar.php:7255-7261`.
- Writing state reads and writes post meta without `edit_post` checks in `includes/class-gutenberg-sidebar.php:823-887`.
- `handle_link_conversation_to_post()` checks target post edit permission, but does not first verify access to the source session in `includes/class-gutenberg-sidebar.php:7453-7482`.
Impact:
- Any user who can edit posts may read or modify state for posts they should not access.
- Session linking can attach another user's known session ID to an editable post.
- Editorial privacy is weak on multi-author sites.
Recommended fix:
- Add `current_user_can('edit_post', $post_id)` to every post-scoped read/write handler.
- Add `current_user_can_access($session_id)` before link-post.
- Add route-level `args` with validation for `post_id` and `session_id`.
### P0: Conversation Table Migration Is Better, But Still Version-fragile
Activation now creates the conversation table, but version checks still mix the main DB version and conversation-specific version. Existing installs with `wpaw_db_version=1.1.0` but no conversation table can still be skipped by `wp_agentic_writer_maybe_create_tables()`.
Evidence:
- Conversation table creation records `wpaw_conversations_db_version` in `includes/class-conversation-migration.php:43-47`.
- `wpaw_run_migrations()` still checks `wpaw_db_version` in `includes/class-conversation-migration.php:67-72`.
- Main table creation only runs when `wpaw_db_version < 1.1.0` in `wp-agentic-writer.php:223-242`.
- Cleanup cron is scheduled at include time in `includes/class-conversation-migration.php:94-99`, independent of table readiness.
Impact:
- A site previously marked upgraded can still miss `wpaw_conversations`.
- Cron may run SQL against a missing table.
- Reinstall and upgrade behavior remains hard to reason about.
Recommended fix:
- Change migration runner to read and update only `wpaw_conversations_db_version`.
- Always call a lightweight `ensure_conversations_table()` on plugin load or before conversation DB access.
- Unschedule `wpaw_cleanup_old_sessions` on deactivation.
## High Priority Findings
### P1: Provider Fallback Violates Provider Transparency Contract
The DoD requires actual provider/model metadata plus warnings in every AI response. Current provider routing still falls back to OpenRouter without surfacing that to the UI.
Evidence:
- DoD provider metadata requirement: `docs/DEFINITION_OF_DONE.md:53-66`.
- Fallback behavior: `includes/class-provider-manager.php:35-50`.
- OpenRouter chat responses include `model`, but not `provider` or `warnings`, in `includes/class-openrouter-provider.php:505-513` and `includes/class-openrouter-provider.php:739-747`.
Impact:
- A user choosing local/private generation may unknowingly send prompts to OpenRouter.
- Sidebar cost/provider labels can be wrong.
- Debugging model routing remains confusing.
Recommended fix:
- Replace raw provider return with a `Provider_Selection_Result` containing provider instance, provider name, selected provider, actual provider, and warnings.
- Include `provider`, `selected_provider`, `fallback_used`, and `warnings` in all AI responses.
- Add a setting for fallback policy: fail closed, ask user, or auto fallback.
### P1: Cost Tracker Is Still a Log, Not a Reliable Ledger
Cost tracking inserts rows, but it does not honor the tracking setting, record failures, record provider/session IDs, or enforce budget limits.
Evidence:
- DoD cost integrity requires success and failed calls to be intentionally recorded in `docs/DEFINITION_OF_DONE.md:68-75`.
- Cost table schema lacks provider/session/status/error columns in `wp-agentic-writer.php:198-209`.
- `WP_Agentic_Writer_Cost_Tracker::add_request()` unconditionally inserts in `includes/class-cost-tracker.php:58-75`.
- Image generation returns variant costs but does not call the ledger hook in `includes/class-image-manager.php:482-522`.
Impact:
- "Cost tracking disabled" is ambiguous because backend records still happen.
- Failed paid attempts are invisible.
- Costs cannot be reconciled by provider or conversation.
Recommended fix:
- Define whether `cost_tracking_enabled` means "show UI" or "store records"; rename or enforce accordingly.
- Add columns: `provider`, `session_id`, `request_id`, `status`, `error_code`, `currency`, `raw_usage`.
- Add preflight estimates and optional hard monthly budget enforcement before remote calls.
### P1: Settings Cost Log Queries Load Too Much Data
The V2 AJAX cost log computes pagination after loading all matching rows. On real sites this can become slow.
Evidence:
- `ajax_get_cost_log_data()` counts rows, then fetches all matching records in `includes/class-settings-v2.php:645-652`.
- It groups in PHP and paginates after formatting in `includes/class-settings-v2.php:654-703`.
Impact:
- Cost log can degrade admin performance as usage grows.
- Large histories increase memory usage and can time out.
Recommended fix:
- Push grouping and pagination into SQL.
- Add indexes for `created_at`, `action`, `model`, and composite reporting queries.
### P1: Writing State Routes Can Leak or Modify Other Users' Draft State
The new writing-state endpoints read and write post meta but rely only on the broad `edit_posts` permission.
Evidence:
- Routes registered at `includes/class-gutenberg-sidebar.php:624-641`.
- Read/write handlers do not call `current_user_can('edit_post', $post_id)` in `includes/class-gutenberg-sidebar.php:823-887`.
Impact:
- A contributor/editor can potentially inspect or change workflow state for a post outside their permissions.
- Pause/resume state can be corrupted across users.
Recommended fix:
- Require `edit_post` for the specific post in both handlers.
- Sanitize status against an allowlist: `idle`, `in_progress`, `paused`, `completed`, `failed`.
- Consider moving writing state into the conversation/session record to reduce post meta sprawl.
## Medium Priority Findings
### P2: Context Service Exists But Is Not the Unified Interface
`WP_Agentic_Writer_Context_Service` is a useful foundation, but most generation paths still assemble context directly in `class-gutenberg-sidebar.php`. Static search shows only chat persistence and migration use it.
Evidence:
- Context Service methods exist in `includes/class-context-service.php`.
- Current references from sidebar are limited to message append and migration in `includes/class-gutenberg-sidebar.php:1009`, `includes/class-gutenberg-sidebar.php:1151`, and `includes/class-gutenberg-sidebar.php:7526`.
- DoD requires all generation paths to use it in `docs/DEFINITION_OF_DONE.md:29-52`.
Opportunity:
- Make `Context_Service::build_ai_context()` the only path for chat, plan, write, refine, SEO/GEO, image prompt generation, and suggestions.
- Let REST handlers pass request data to Workflow Service, not directly assemble prompts.
### P2: Session IDs Are Short and Non-cryptographic
Session IDs are generated with `substr(md5(uniqid(wp_rand(), true)), 0, 16)`.
Evidence:
- `includes/class-conversation-manager.php:63-65`
Impact:
- The attack surface is still small, but this is weaker than modern token generation.
- Since session IDs gate private editorial context, stronger IDs are cheap insurance.
Recommended fix:
- Use `bin2hex(random_bytes(16))` where available with a WP fallback, or `wp_generate_uuid4()`.
- Expand DB column from `VARCHAR(32)` if needed.
### P2: Model Defaults Are Still Fragmented
The implementation made some defaults cheaper, but there is still no single model registry.
Evidence:
- Activation still stores `planning_model` and legacy `execution_model` in `wp-agentic-writer.php:140-142`.
- Provider defaults differ in `includes/class-openrouter-provider.php:34-69`.
- Settings V2 has multiple fallback sets in `includes/class-settings-v2.php:105-111`, `includes/class-settings-v2.php:228-273`, and `includes/class-settings-v2.php:990-995`.
- JS has another set in `assets/js/settings-v2.js:26-38`.
Impact:
- Fresh install, upgraded install, saved settings, and JS reset can produce different model choices.
- Support/debugging becomes harder because "default" depends on code path.
Recommended fix:
- Add `WP_Agentic_Writer_Model_Registry`.
- Expose the registry to JS through localization.
- Migrate `execution_model` to `writing_model` and stop storing both.
### P2: Debug Logging Is Partly Gated, Frontend Logging Is Not
Backend logging improved in some places, but frontend debug logs remain numerous and some backend providers still log operational details.
Evidence:
- `wpaw_debug_log()` is gated in `includes/class-gutenberg-sidebar.php:20-27`.
- Frontend migration/session/clarity logs remain in `assets/js/sidebar.js:308-322`, `assets/js/sidebar.js:5619-5630`, and `assets/js/sidebar.js:6130-6157`.
- Settings debug logs remain in `assets/js/settings-v2.js:53-130` and cost log logs in `assets/js/settings-v2.js:390-503`.
Impact:
- Browser console can expose topics, local backend behavior, model state, and debug-only workflows.
- Debug noise hides real errors for users.
Recommended fix:
- Add `wpAgenticWriter.debug` and `wpawSettingsV2.debug`.
- Wrap all console logging behind a tiny logger utility.
- Keep `console.error` only for actionable user-visible failures.
### P2: Admin UX Still Has External Runtime Dependencies
Settings V2 still uses CDN Bootstrap and Select2.
Evidence:
- `includes/class-settings-v2.php:67-75`
Impact:
- Settings UI can break offline, under CSP, or in restricted enterprise admin environments.
- External admin CDNs create privacy and supply-chain risk.
Recommended fix:
- Bundle vendor files locally or rebuild the settings UI with WordPress components.
### P2: Uninstall and Data Lifecycle Are Not Clean
The main uninstall hook and `uninstall.php` disagree. Neither fully removes conversations, versions, custom models, all post meta, transients, or cron events.
Evidence:
- Main uninstall: `wp-agentic-writer.php:269-294`
- Separate uninstall file: `uninstall.php:12-21`
Impact:
- Reinstalls can inherit stale model settings, chat sessions, and DB versions.
- Testing fresh install behavior remains unreliable.
Recommended fix:
- Choose one uninstall path.
- Add an admin setting for "delete all plugin data on uninstall".
- Delete all plugin options, transients, scheduled hooks, tables, upload temp files, and known post/user meta.
## UI/UX Opportunities
### Make Context Visible
Users need a compact "What the agent knows" panel: active session, linked post, focus keyword, language, plan status, message count, provider, model, and estimated cost. This directly reduces the confusion when a conversation resumes with unexpected memory.
### Make Provider Execution Explicit
Before any paid/private-sensitive action, show the selected provider and the actual provider health. If fallback will happen, ask or show a visible warning. After the call, show the actual provider/model used.
### Add Workflow State Instead of Hidden Modes
The sidebar has chat, sessions, planning, writing, cost, SEO, clarification, resume, and suggestions. These should map to one visible state machine:
1. Context
2. Plan
3. Write
4. Review
5. Publish Assist
Each state should have one clear primary action and one clear recovery action.
### Add Review/Accept Safety for High-impact Edits
Generated blocks and refinements should have a review layer for diff/accept/reject, especially for article-wide and multi-pass edits.
## Definition of Done Compliance
| DoD area | Current compliance | Notes |
|---|---:|---|
| Storage layer declaration | Partial | DoD exists, but current code still uses dual chat storage. |
| Context Service usage | Failing | Not all generation paths use Context Service. |
| Provider transparency | Failing | Responses do not consistently include `provider` or fallback warnings. |
| Cost record integrity | Failing | Failed calls and image calls are not consistently ledgered. |
| Workflow tests | Unknown | Syntax checks pass, but no end-to-end workflow evidence found. |
| No double source of truth | Failing | Conversation messages still exist in post meta and sessions. |
| Migration safety | Partial | Conversation table creation added, but versioning remains inconsistent. |
| Security contract | Partial | Some session endpoints fixed, post-scoped routes remain broad. |
| Changelog policy | Failing | `CHANGELOG.md` was not found while DoD requires it. |
## Recommended Next Work Queue
### Do First
1. Split OpenRouter model cache keys and flush old transient data.
2. Enforce `edit_post` and session access checks on all post/session routes.
3. Make conversation messages session-table only for new writes.
4. Fix conversation migration versioning to use `wpaw_conversations_db_version` independently.
### Do Second
1. Implement provider selection metadata and fallback warnings.
2. Upgrade cost tracking into a provider/session-aware ledger.
3. Centralize model defaults in a registry.
4. Gate frontend and backend debug logging.
### Do Third
1. Extract REST controllers and workflow services from `class-gutenberg-sidebar.php`.
2. Bundle admin dependencies locally.
3. Consolidate uninstall/data retention behavior.
4. Add end-to-end workflow tests for Chat -> Plan -> Write, Stop -> Resume, and Clear Context -> New Plan.
## Completion Marker
The original 2026-05-22 audit is now marked complete/superseded. Remaining work should be tracked from this follow-up audit only, to avoid duplicate jobs debt.

526
docs/architecture/PLUGIN_AUDIT_REPORT_2026-05-22.md Normal file
View File

@@ -0,0 +1,526 @@
# WP Agentic Writer Plugin Audit Report
Status: COMPLETE / SUPERSEDED
Completion marker date: 2026-05-24
Follow-up trace audit: `docs/architecture/PLUGIN_AUDIT_FOLLOWUP_2026-05-24.md`
This report is retained as the historical baseline. Its implementation has been traced in the 2026-05-24 follow-up audit, so remaining work should be tracked from the follow-up report instead of reopening duplicate jobs from this file.
Audit date: 2026-05-22
Plugin version observed: 0.1.3
Scope: UI, UX, admin settings, Gutenberg sidebar workflow, conversation context/history, cost tracking, provider/model routing, image generation, local backend, security, data lifecycle, maintainability.
## Executive Summary
WP Agentic Writer has a strong product direction: a plan-first writing assistant inside Gutenberg with chat, planning, writing, refinement, research, image suggestions, SEO/GEO helpers, provider routing, local backend support, and cost visibility. The problem is not lack of ambition. The problem is that too many responsibilities are packed into a few files without stable contracts between state, persistence, providers, and UI.
The highest risk pattern is this: the plugin now has two overlapping persistence models for conversation history. Older post meta storage (`_wpaw_chat_history`, `_wpaw_plan`, `_wpaw_memory`) still exists, while newer session storage (`wpaw_conversations`) was added but is not reliably migrated or permission-scoped. That creates the exact failure mode you described: fixing one flow can silently break another because different screens and endpoints read from different truth sources.
Overall readiness assessment: beta/prototype with several production blockers. Syntax checks pass for key PHP and JS files, but there are serious runtime, migration, security, and model-cache defects.
## Critical Findings
### P0: Conversation Table Migration Is Not Wired
The new conversation manager expects a `wpaw_conversations` table, but activation only creates cost and image tables. `wpaw_run_migrations()` exists but is not called. Worse, the general DB version is set to `1.1.0`, while the conversation migration checks for `< 0.1.4`, so sites can be marked upgraded without the conversation table ever being created.
Evidence:
- `wp-agentic-writer.php:136-180` creates default options, custom models, cost table, and image tables, but not conversations.
- `wp-agentic-writer.php:219-231` updates `wpaw_db_version` to `1.1.0`.
- `includes/class-conversation-migration.php:17-44` defines conversation table creation.
- `includes/class-conversation-migration.php:64-69` defines migration runner, but it is not hooked or called.
Impact:
- New chat/session UX can fail with DB insert/read errors on clean installs or upgraded installs.
- Fixing frontend session behavior may appear broken because the database contract is missing.
Recommendation:
- Split DB versions per table/domain, for example `wpaw_cost_db_version`, `wpaw_image_db_version`, `wpaw_conversation_db_version`.
- Call conversation migrations on activation and on `plugins_loaded` idempotently.
- Add a visible admin health check that verifies required tables exist.
### P0: OpenRouter Model Cache Has Conflicting Shapes
`get_cached_models()` stores the full OpenRouter model objects in transient `wpaw_openrouter_models`. `validate_model_availability()` uses the same transient key but expects a flat list of model IDs. If the settings page has already cached full model objects, streaming and image validation will reject valid models because `in_array($model_id, $available_models, true)` compares a string to arrays.
Evidence:
- `includes/class-openrouter-provider.php:105-172` caches full model objects under `wpaw_openrouter_models`.
- `includes/class-openrouter-provider.php:239-255` reads the same transient key as if it contains IDs.
- Validation is used before streaming and image generation at `includes/class-openrouter-provider.php:548` and `includes/class-openrouter-provider.php:755`.
Impact:
- Valid models can fail as "not available".
- Refreshing the model list in settings can break generation.
- This creates a brittle A/B loop: model UI fixes can break streaming/image execution.
Recommendation:
- Use separate cache keys, e.g. `wpaw_openrouter_model_objects` and `wpaw_openrouter_model_ids`.
- Normalize model validation to accept both canonical IDs and suffix variants like `:online`, without poisoning the settings model cache.
- Add a regression test around cached full model objects plus streaming validation.
### P0: PHP Requirement Is 7.4 But Code Uses PHP 8 Functions
The plugin header declares PHP 7.4 support, but the provider streaming parsers call `str_starts_with()`, which requires PHP 8.
Evidence:
- `wp-agentic-writer.php:13-14` declares `Requires PHP: 7.4`.
- `includes/class-openrouter-provider.php:642`, `includes/class-local-backend-provider.php:208`, and `includes/class-codex-provider.php:207` call `str_starts_with()`.
Impact:
- Fatal errors on PHP 7.4 sites when streaming code paths load.
Recommendation:
- Either raise `Requires PHP` to 8.0+ or replace with `0 === strpos($line, 'data: ')`.
### P0: Conversation Endpoints Lack Per-Session Ownership Checks
REST permission is only `current_user_can('edit_posts')`. The conversation handlers read, update, delete, and overwrite messages by `session_id` without checking that the session belongs to the current user or that the user can edit the linked post.
Evidence:
- `includes/class-gutenberg-sidebar.php:847-849` grants all REST routes to anyone who can edit posts.
- `includes/class-gutenberg-sidebar.php:6977-6991` returns any session by `session_id`.
- `includes/class-gutenberg-sidebar.php:7001-7033` updates any session by `session_id`.
- `includes/class-gutenberg-sidebar.php:7043-7060` deletes any session by `session_id`.
- `includes/class-gutenberg-sidebar.php:7070-7098` overwrites messages for any session by `session_id`.
Impact:
- Any editor-level user who obtains or guesses a session ID can read or modify another user's conversation.
- Stored article prompts, SEO keywords, unpublished plans, and drafts can leak.
Recommendation:
- Add `Conversation_Manager::current_user_can_access($session_id)` and enforce it on all session routes.
- For linked post sessions, also require `current_user_can('edit_post', $post_id)`.
- Increase session IDs to a stronger token, e.g. `wp_generate_uuid4()` or `bin2hex(random_bytes(16))`.
## High Priority Findings
### P1: Two Context Stores Compete Instead Of Cooperating
Current code keeps post meta chat history and new session messages at the same time.
Evidence:
- `handle_chat_request()` updates post meta chat history at `includes/class-gutenberg-sidebar.php:924-930` and later.
- Frontend saves every message array to `/conversations/{session_id}/messages` at `assets/js/sidebar.js:287-318`.
- Frontend initializes sessions through `/conversations/post/{postId}` and `/conversations?uncompleted=1` at `assets/js/sidebar.js:192-267`.
Impact:
- Chat mode, planning mode, writing mode, and resume mode can see different histories.
- Clearing context deletes post meta but not necessarily the session messages.
- "Continue conversation" can restore messages while `_wpaw_plan` or `_wpaw_memory` remains stale.
Recommendation:
- Pick one source of truth for conversational history. Prefer `wpaw_conversations` for messages and context, with post meta only storing the current plan and lightweight indexes.
- Define a single context assembly service used by chat, plan, write, refine, SEO, and image flows.
- Make "clear context" clear both the active session messages/context and legacy post meta during migration.
### P1: Provider Routing Falls Back Silently To OpenRouter
If a configured local backend is unreachable or unsupported, provider manager silently falls back to OpenRouter.
Evidence:
- `includes/class-provider-manager.php:33-45` returns OpenRouter fallback if selected provider is not configured or local connection test fails.
Impact:
- A user choosing local/private/free generation may unknowingly send prompts to OpenRouter.
- Cost expectations and privacy expectations can be violated.
- Debugging provider behavior becomes confusing because UI selection is not guaranteed execution.
Recommendation:
- Make fallback behavior explicit and configurable: "fail closed" vs "fallback to OpenRouter".
- Return provider metadata in each API response so the UI can show the actual provider used.
- Add a preflight provider health state in settings and sidebar.
### P1: Cost Tracking Setting Does Not Stop Tracking Or Enforce Budget
`cost_tracking_enabled` controls parts of the frontend display, but the backend cost hook always writes records. Monthly budget is display-only and does not prevent expensive calls.
Evidence:
- Cost tracker always registers the hook at `includes/class-cost-tracker.php:42-44`.
- `add_request()` inserts every event without checking settings at `includes/class-cost-tracker.php:58-75`.
- Frontend skips fetching if disabled at `assets/js/sidebar.js:501-505`, but backend still records.
Impact:
- The setting name implies disabling tracking, but data is still stored.
- Budget UI can be misleading because it is not a guardrail.
Recommendation:
- Decide whether the setting means "hide UI" or "do not store usage"; rename or implement accordingly.
- Add optional soft and hard budget policies before provider calls.
- Track actual provider, request ID, session ID, and failure state for reconciliation.
### P1: API Route Contracts Are Too Loose
Most REST routes accept raw JSON and manually read fields. Routes do not declare `args` schemas or sanitize/validate centrally.
Evidence:
- Routes are registered without `args` schemas beginning at `includes/class-gutenberg-sidebar.php:287-365`.
- Handler code manually reads arbitrary payloads, e.g. `handle_chat_request()` at `includes/class-gutenberg-sidebar.php:858-914`.
Impact:
- Small frontend changes can break backend assumptions.
- Security review becomes harder because validation is spread across handlers.
- No machine-readable contract exists for tests.
Recommendation:
- Add route `args` definitions for all simple endpoints.
- Introduce request DTO/helper methods for complex generation/refinement requests.
- Add contract tests for each endpoint with valid, missing, malformed, and unauthorized payloads.
### P1: Main Backend Class Is Too Large To Change Safely
`includes/class-gutenberg-sidebar.php` is roughly 7,200 lines and owns asset enqueueing, route registration, request validation, prompt assembly, streaming, SEO, GEO, research, image routes, conversation routes, and persistence.
Impact:
- Any change has a large blast radius.
- Prompt changes, UI changes, and persistence changes are tangled.
- This directly contributes to "fix A, lose B" cycles.
Recommendation:
- Split by ownership:
- `Rest_Routes` registers routes only.
- `Context_Service` assembles messages/context/history.
- `Workflow_Service` handles planning/writing/refinement state.
- `Provider_Service` wraps provider selection and fallback.
- `Cost_Service` handles usage policies.
- `Conversation_Rest_Controller`, `Image_Rest_Controller`, `Seo_Rest_Controller`.
## Medium Priority Findings
### P2: Admin Settings Depend On External CDNs
The settings page enqueues Bootstrap and Select2 from CDN.
Evidence:
- `includes/class-settings-v2.php:67-75` loads CDN CSS/JS.
Impact:
- Settings UI can break offline or in restricted admin environments.
- Supply-chain and privacy expectations are weaker for a plugin admin page.
Recommendation:
- Bundle vendor assets locally or use WordPress-native components where possible.
### P2: Uninstall Is Incomplete And Duplicated
There is both `register_uninstall_hook()` in the main plugin file and an `uninstall.php`. Cleanup differs between them and neither fully cleans new data.
Evidence:
- Main uninstall deletes settings and cost/image tables at `wp-agentic-writer.php:259-267`.
- `uninstall.php` deletes settings, `_wpaw_plan`, and cost table only.
- Neither path deletes `wp_agentic_writer_custom_models`, `wpaw_db_version`, `wpaw_conversations`, `_wpaw_chat_history`, `_wpaw_memory`, `_wpaw_post_config`, `_wpaw_detected_language`, writing state meta, or image-related post meta.
Impact:
- Reinstall behavior is unpredictable.
- Old settings and tables can affect fresh testing.
Recommendation:
- Use one uninstall path.
- Add a documented "delete all data on uninstall" option.
- Clean all plugin options, transients, tables, upload temp files, scheduled events, and post meta.
### P2: Image Generation Is Partially Integrated
The image manager has tables, recommendations, variants, commit flow, and temp cleanup, but cost tracking and error handling are incomplete.
Risks:
- Image generation costs are not consistently inserted into the cost tracking table.
- Temp files are written with `file_put_contents()` without checking result or validating MIME/content length.
- Committed variants use `media_handle_sideload()` from the temp path, so failure modes can delete/move temp files unexpectedly.
Recommendation:
- Add `wp_aw_after_api_request` events for image generation.
- Validate downloaded image type and size before writing.
- Add image state transitions: pending -> generating -> temp_ready -> committed -> failed.
### P2: Settings Defaults And Model Labels Are Inconsistent
Defaults differ across activation, settings V2, OpenRouter provider, settings fallback, and UI copy.
Examples:
- Activation uses `execution_model` but current code uses `writing_model`.
- Activation default planning model is `google/gemini-2.0-flash-exp`, while settings/provider defaults use `google/gemini-2.5-flash`.
- Refinement defaults vary between Haiku and Sonnet.
Impact:
- Fresh install, upgraded install, and settings save can select different models.
- Model bugs are hard to reproduce because initial state depends on install path.
Recommendation:
- Create a single model preset registry in PHP and expose it to JS.
- Run one migration that maps `execution_model` to `writing_model` and removes stale defaults.
- Add "current saved model is unavailable" UI with fallback choice.
### P2: Debug Logging Is Too Noisy For Production
Several `error_log()` and `console.log()` calls are unconditional or reveal request behavior and settings.
Examples:
- Asset enqueue logs at `includes/class-gutenberg-sidebar.php:73-74`.
- Provider routing logs at `includes/class-provider-manager.php:28`.
- Streaming provider settings logs at `includes/class-gutenberg-sidebar.php:3041-3042`.
- Frontend session logs at `assets/js/sidebar.js:5119-5130`.
Impact:
- Logs can expose topics, model choices, local backend status, and partial AI responses.
- Debug noise hides real defects.
Recommendation:
- Add `wpaw_debug_log()` gated behind `WP_DEBUG && SCRIPT_DEBUG` or a plugin debug setting.
- Never log API keys, full prompts, full responses, or private drafts by default.
## UI/UX Assessment
### What Works
- The product concept is coherent: chat -> clarify -> plan -> write -> refine.
- Gutenberg-side integration is stronger than a typical "AI text box" plugin.
- @mentions and block toolbar actions are a strong foundation for an IDE-like writing workflow.
- The admin settings V2 layout gives a clearer mental model for model selection, local backend, cost analytics, and docs.
### UX Gaps
- The sidebar has too many implicit modes. Users can be in chat, planning, writing, sessions list, welcome screen, empty writing state, cost tab, SEO tab, and clarification mode, but those states do not share a single state machine.
- "Writing mode" can behave like discussion-only in some paths, while actual writing requires a plan. This is easy to misunderstand.
- Context status is not transparent enough. Users cannot easily see "what the agent remembers", "which session is active", "which provider will run", or "what will be sent".
- Cost UI shows spend, but not clear preflight estimates or post-call reconciliation by provider.
- There is no review/accept/reject safety layer for high-impact article edits. Generated blocks can be inserted directly.
### Recommended UX Direction
Replace mode ambiguity with a visible workflow state:
1. Context: topic, keyword, language, audience, source material.
2. Plan: outline draft, editable sections, approve plan.
3. Write: section-by-section generation with pause/resume.
4. Review: diff, SEO/GEO checks, image recommendations.
5. Publish assist: metadata, schema, final checklist.
Each state should expose the active provider, cost estimate, context source, and next best action.
## System Architecture Assessment
### Current Shape
```mermaid
flowchart TD
UI["assets/js/sidebar.js"]
Routes["class-gutenberg-sidebar.php"]
OR["OpenRouter Provider"]
Local["Local Backend Provider"]
Codex["Codex Provider"]
Cost["Cost Tracker"]
Meta["Post Meta"]
Conv["wpaw_conversations"]
Images["Image Manager"]
UI --> Routes
Routes --> OR
Routes --> Local
Routes --> Codex
Routes --> Cost
Routes --> Meta
Routes --> Conv
Routes --> Images
UI --> Conv
UI --> Meta
```
The core issue is that both UI and backend understand too much about everything. The architecture needs boundaries more than it needs new features.
### Target Shape
```mermaid
flowchart TD
UI["Sidebar UI"]
REST["REST Controllers"]
Workflow["Workflow Service"]
Context["Context Service"]
Provider["Provider Gateway"]
Cost["Cost Policy + Ledger"]
Store["Conversation + Post State Store"]
UI --> REST
REST --> Workflow
Workflow --> Context
Workflow --> Provider
Workflow --> Cost
Context --> Store
Cost --> Store
```
The important change is that every generation path asks the same `Context_Service` for context and the same `Provider_Gateway` for provider execution. That gives you one place to fix context bugs and one place to fix provider/cost bugs.
## Context And History Audit
Current context layers:
- Frontend React state: immediate but volatile.
- `localStorage`: agent mode only.
- Post meta: `_wpaw_chat_history`, `_wpaw_plan`, `_wpaw_memory`, `_wpaw_post_config`, `_wpaw_detected_language`, writing state.
- Conversation table: session messages/context/status/title/focus keyword.
Key gaps:
- Session context field exists but frontend mostly saves messages, not a normalized workflow context.
- Post-linked and uncompleted sessions are mixed into the same UI without a clear transition.
- Auto-save of every messages array can overwrite richer backend state with stale frontend state.
- There is no schema/version for message objects, so plan cards, timeline entries, assistant messages, and system info live in the same array.
Recommended contract:
```json
{
"session_id": "uuid",
"post_id": 123,
"workflow_state": "context|planning|writing|review|done",
"messages": [],
"context_summary": "",
"plan_id": "uuid",
"active_provider": "openrouter|local_backend|codex",
"cost_session_id": "uuid",
"updated_at": "datetime"
}
```
## Cost Tracking Audit
Current strengths:
- Central cost hook exists.
- Sidebar and settings cost views exist.
- Cost log grouping by post is useful.
Current gaps:
- No session ID in cost records.
- No provider column.
- No request status or error records.
- No distinction between estimated and actual cost.
- No hard budget stop.
- Disabled tracking does not stop backend inserts.
- Local backend and Codex cost semantics differ from OpenRouter but share the same table model.
Recommended table changes:
- `provider`
- `session_id`
- `request_id`
- `status`
- `estimated_cost`
- `actual_cost`
- `currency`
- `metadata_json`
## Models And Provider Audit
Current strengths:
- Per-task model selection is directionally right.
- OpenRouter model refresh exists.
- Custom models can be added.
- Provider routing supports OpenRouter, local backend, and Codex.
Current gaps:
- Model cache bug is production-blocking.
- Provider fallback is silent.
- Codex provider uses older Chat Completions assumptions and hardcoded stale pricing.
- Local backend test runs an inference call, which may be unexpectedly slow/costly for a "test connection".
- Image model selection trusts OpenRouter modalities but custom models bypass capability validation.
Recommended provider contract:
```php
ProviderResult {
provider: string,
model: string,
content: string,
usage: Usage,
cost: Cost,
capabilities: string[],
warnings: string[]
}
```
## Test And Verification Gaps
Checks run during this audit:
- `php -l wp-agentic-writer.php`
- `php -l includes/class-gutenberg-sidebar.php`
- `php -l includes/class-settings-v2.php`
- `php -l includes/class-openrouter-provider.php`
- `php -l includes/class-image-manager.php`
- `php -l includes/class-conversation-migration.php`
- `node --check assets/js/sidebar.js`
- `node --check assets/js/settings-v2.js`
- `node --check assets/js/sidebar-utils.js`
- `node --check assets/js/block-refine.js`
- `node --check assets/js/block-image-generate.js`
All checked files passed syntax checks.
Missing test coverage:
- Activation/migration tests for clean install and upgrade.
- REST permission tests for conversations and post config.
- Provider model-cache regression tests.
- Context assembly snapshots per mode.
- Streaming parser tests for OpenRouter, local backend, and Codex.
- Cost ledger tests with tracking disabled, zero-cost local calls, and failed requests.
- Gutenberg e2e tests for chat -> plan -> write -> refresh -> resume.
## Stabilization Roadmap
### Phase 1: Stop Runtime Breakage
1. Fix PHP 7.4 compatibility or raise PHP requirement.
2. Fix OpenRouter model cache shape conflict.
3. Wire conversation migrations correctly.
4. Add ownership checks on all conversation endpoints.
5. Gate debug logging.
### Phase 2: Stabilize State
1. Declare one source of truth for conversation messages.
2. Create a context service used by all generation paths.
3. Migrate legacy post meta chat history into sessions.
4. Make clear context/session/post behavior explicit.
5. Add workflow state to session context.
### Phase 3: Stabilize Cost And Provider Behavior
1. Add provider metadata to all AI responses.
2. Make provider fallback explicit.
3. Add budget preflight and optional hard limit.
4. Expand cost table with provider/session/request fields.
5. Track image and failed request costs consistently.
### Phase 4: Reduce Blast Radius
1. Split `class-gutenberg-sidebar.php` into controllers and services.
2. Add REST schemas and shared request validators.
3. Build integration tests around the main workflows.
4. Add a small internal fixture suite for model/provider responses.
5. Remove backup files and duplicate settings/documentation paths after confirming they are unused.
## Highest Leverage Opportunities
- Make the plugin feel safer: add preview/diff/accept/reject for refinements and article-wide edits.
- Make the agent feel smarter: show "current context" and let users edit what the agent remembers.
- Make costs trustworthy: show preflight estimate, actual cost, provider, and model after every operation.
- Make local backend trustworthy: no silent cloud fallback unless the user explicitly opts in.
- Make model selection resilient: capability badges, availability checks, and clear fallbacks.
- Make the codebase easier to evolve: services plus tests around the workflows that matter.
## Suggested Definition Of Done For Future Fixes
For any feature or bug fix touching chat, planning, writing, refinement, context, provider, or cost:
1. It must state which storage layer is authoritative.
2. It must include the provider/model actually used in the response.
3. It must update or preserve cost records intentionally.
4. It must pass at least one workflow test from chat to final editor state.
5. It must not add another source of truth for the same state.
This is the guardrail that prevents losing A while fixing B.

276
docs/architecture/PLUGIN_AUDIT_RETRACE_2026-05-24.md Normal file
View File

@@ -0,0 +1,276 @@
# WP Agentic Writer Retrace Audit
Status: COMPLETE / SUPERSEDED
Completion marker date: 2026-05-24
Next retrace report: `docs/architecture/PLUGIN_AUDIT_RETRACE_SECOND_PASS_2026-05-24.md`
Audit date: 2026-05-24
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_FOLLOWUP_2026-05-24.md`
Scope: current implementation after follow-up fixes, with emphasis on UI/UX, system boundaries, conversation context/history, cost tracker, provider/model routing, migrations, and data lifecycle.
## Executive Summary
The follow-up implementation closed several important items, but the plugin is not yet clean enough to shift only into chat/context implementation. The highest-risk remaining problem is a new provider contract mismatch: `WP_Agentic_Writer_Provider_Manager::get_provider_for_task()` now returns a `WPAW_Provider_Selection_Result`, but several older classes still treat the return value as a provider and call `->chat()` or `->generate_image()` directly. That can fatal in image, keyword, and WP AI wrapper paths.
The second urgent issue is streaming chat state: `stream_chat_request()` currently references `$accumulated_content`, `$chunks_emitted`, and `$last_user_message` without initializing them. That can corrupt streaming persistence and produce warnings or missing session messages.
Good news: OpenRouter cache separation is now implemented, several conversation permissions were tightened, settings cost-log pagination was improved, and chat responses now include provider metadata. But there are still cross-cutting runtime, migration, cost-ledger, and authorization gaps.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- JS syntax check: `node -c assets/js/sidebar.js` and `node -c assets/js/settings-v2.js` passed.
- Static trace of follow-up audit items against current code.
- No live WordPress browser workflow was run in this pass.
## Follow-up Status Trace
| Follow-up item | Current status | Evidence |
|---|---:|---|
| Split OpenRouter model cache keys | Fixed | Full objects use `wpaw_openrouter_model_objects`; IDs use `wpaw_openrouter_model_ids` in `includes/class-openrouter-provider.php:106-264`. |
| Add post/session auth for conversation list/create/link | Mostly fixed | `edit_post` added for post-linked conversation routes and link-post checks session access in `includes/class-gutenberg-sidebar.php:7263-7555`. |
| Add writing-state post auth | Fixed | `edit_post` checks and status allowlist added in `includes/class-gutenberg-sidebar.php:823-887`. |
| Provider fallback metadata for chat | Partially fixed | Provider result object and chat response metadata exist in `includes/class-provider-manager.php:20-92` and `includes/class-gutenberg-sidebar.php:987-1029`, but helper classes still use the old API. |
| Cost log SQL pagination | Improved | Grouping/pagination moved into SQL in `includes/class-settings-v2.php:645-667`. |
| Conversation table migration versioning | Still open | Migration still checks `wpaw_db_version`, not `wpaw_conversations_db_version`, in `includes/class-conversation-migration.php:67-72`. |
| Single source of truth for chat messages | Partially fixed | Chat no longer writes new `_wpaw_chat_history`, but legacy methods/routes remain and migration does not delete migrated meta. |
| Cost tracker provider/session/status ledger | Partially fixed | New columns and parameters exist, but hook accepts only 7 args in `includes/class-cost-tracker.php:48-53`, so session/status are dropped. |
| Model registry/default unification | Still open | Defaults remain fragmented across activation, providers, settings PHP, JS, and WP AI wrapper. |
| Uninstall/data lifecycle | Still open | Main uninstall and `uninstall.php` remain inconsistent and incomplete. |
| Debug logging | Still open | Backend and frontend debug logs remain broad. |
## Critical Findings
### P0: Provider Manager Return Contract Breaks Older Callers
`get_provider_for_task()` now returns `WPAW_Provider_Selection_Result`, which is correct for provider transparency. But not every caller was updated to use `$provider_result->provider`.
Broken paths:
- Image placement analysis calls `$provider->chat()` on the selection result in `includes/class-image-manager.php:218-219`.
- Image prompt generation does the same in `includes/class-image-manager.php:295-296`.
- Image variant generation calls `$provider->generate_image()` on the selection result in `includes/class-image-manager.php:478-483`.
- Keyword suggester calls `$provider->chat()` on the selection result in `includes/class-keyword-suggester.php:32-84`.
- WP AI legacy wrapper calls `$provider->chat()`, `$provider->chat_stream()`, and `$provider->get_name()` on the selection result in `includes/class-wp-ai-client-wrapper.php:225-252` and `includes/class-wp-ai-client-wrapper.php:272-280`.
Impact:
- Image generation, image prompt analysis, keyword suggestions, and WP AI fallback paths can fatal with "Call to undefined method WPAW_Provider_Selection_Result::chat()".
- This is a classic fix-A-break-B regression from changing a shared service contract.
Recommended fix:
- Update all callers to:
- `$provider_result = WP_Agentic_Writer_Provider_Manager::get_provider_for_task( ... );`
- `$provider = $provider_result->provider;`
- propagate `$provider_result->actual_provider`, `fallback_used`, and `warnings` where returned or tracked.
- Consider adding `__call()` only as a temporary compatibility shim if too many callers exist, but explicit updates are safer.
### P0: Streaming Chat Uses Uninitialized State
`stream_chat_request()` references variables that are never initialized in the current implementation.
Evidence:
- Closure captures `$accumulated_content` and `$chunks_emitted` by reference in `includes/class-gutenberg-sidebar.php:1089-1109`.
- `$chunks_emitted` is compared at `includes/class-gutenberg-sidebar.php:1111-1121`.
- `$accumulated_content` is used at `includes/class-gutenberg-sidebar.php:1158-1164`.
- `$last_user_message` is stored into session messages at `includes/class-gutenberg-sidebar.php:1169-1177`, but is never assigned inside the method.
Impact:
- Streaming chat can emit warnings/notices and fail to persist the user message correctly.
- Session history may save an empty or undefined user message while the assistant message is stored.
- This directly affects the chat/context path the team wants to focus on.
Recommended fix:
- Restore initializers at the top of `stream_chat_request()`:
- `$accumulated_content = '';`
- `$chunks_emitted = 0;`
- `$last_user_message = $this->get_last_user_message( $messages );`
- `$total_cost = 0;`
### P0: Image Generation Is Broken By Provider Contract Mismatch
Image generation was already a sensitive flow because it touches model routing, temporary files, cost, and DB state. The provider result contract now breaks it before generation.
Evidence:
- `generate_image_variants()` assigns provider result to `$provider` at `includes/class-image-manager.php:478`.
- It calls `$provider->generate_image()` at `includes/class-image-manager.php:483`.
Impact:
- `/generate-image` can fail at runtime even though PHP syntax passes.
- Image generation costs still will not reliably reach the main cost ledger because generation is interrupted before any ledger call.
Recommended fix:
- Unwrap provider result and track image generation through `wp_aw_after_api_request` with provider/session/status metadata.
## High Priority Findings
### P1: Cost Tracker Hook Drops Session and Status Metadata
The Cost Tracker now accepts provider, session ID, and status, but the hook registration only allows 7 arguments.
Evidence:
- Hook registration accepts 7 args in `includes/class-cost-tracker.php:48-53`.
- `add_request()` signature expects 9 args in `includes/class-cost-tracker.php:113-133`.
- Chat emits provider, session ID, and status in `includes/class-gutenberg-sidebar.php:1011-1023` and `includes/class-gutenberg-sidebar.php:1144-1156`.
Impact:
- Provider may be recorded, but `session_id` and explicit `status` are silently dropped.
- Failed calls still are not intentionally recorded.
- Cost records cannot be reconciled to conversation context.
Recommended fix:
- Change hook registration to `add_action( 'wp_aw_after_api_request', array( $this, 'add_request' ), 10, 9 );`.
- Add explicit failure ledger entries around provider errors.
- Add `error_code`/`request_id` if this is meant to be a true ledger.
### P1: Conversation Migration Is Still Version-fragile
The follow-up audit recommended conversation-specific migration versioning, but the migration runner still checks the main DB version.
Evidence:
- Conversation table creation stores `wpaw_conversations_db_version` in `includes/class-conversation-migration.php:43-47`.
- `wpaw_run_migrations()` still reads `wpaw_db_version` in `includes/class-conversation-migration.php:67-72`.
- Main table creation remains gated by `wpaw_db_version < 1.1.0` in `wp-agentic-writer.php:223-241`.
- Conversation cleanup cron is scheduled at include time in `includes/class-conversation-migration.php:94-99`.
Impact:
- Existing installs with `wpaw_db_version=1.1.0` but no conversation table can still be skipped.
- Cleanup cron can run against a missing table.
Recommended fix:
- Run an idempotent `wpaw_ensure_conversations_table()` based on `wpaw_conversations_db_version` or direct table existence.
- Unschedule `wpaw_cleanup_old_sessions` on deactivation.
### P1: Chat History Is Better, But Legacy Read/Migration Still Keeps Two Truths Alive
New chat writes no longer call `update_post_chat_history()`, which is good. But legacy post-meta read/write helpers and migration behavior still keep `_wpaw_chat_history` alive.
Evidence:
- New chat comments say legacy meta is deprecated in `includes/class-gutenberg-sidebar.php:1031-1053` and `includes/class-gutenberg-sidebar.php:1167-1187`.
- `/chat-history/{post_id}` still reads legacy meta in `includes/class-gutenberg-sidebar.php:1240-1256`.
- Legacy update helper still exists and writes meta in `includes/class-gutenberg-sidebar.php:1268-1301`.
- Context Service `get_context()` does not migrate on read in `includes/class-context-service.php:62-87`.
- Migration still leaves legacy meta in place in `includes/class-context-service.php:299-300`.
Impact:
- Legacy history can still be surfaced, migrated more than once, or diverge from the session table.
- Clear context does not clear an active session unless another caller invokes `Context_Service::clear_context()` with a session ID.
Recommended fix:
- Make `/chat-history/{post_id}` migration-only or remove it after frontend no longer needs it.
- Delete or mark `_wpaw_chat_history` after successful migration.
- Call migration from `Context_Service::get_context()` when legacy meta exists.
- Update `handle_clear_context()` to require `edit_post` and clear active session messages when `sessionId` is present.
### P1: Post-scoped Authorization Remains Incomplete Outside Conversation Routes
Conversation routes improved, but other post-scoped routes still trust the broad `edit_posts` permission callback.
Examples:
- Post config get/update read and write `_wpaw_post_config` without `edit_post` in `includes/class-gutenberg-sidebar.php:1722-1756`.
- Cost tracking for a post lacks a target post check in `includes/class-gutenberg-sidebar.php:3623-3629`.
- Section block mapping reads/writes post meta without `edit_post` in `includes/class-gutenberg-sidebar.php:4776-4835`.
- Image recommendation/generate/commit routes are registered with broad permissions at `includes/class-gutenberg-sidebar.php:593-620` and handlers lack target post checks in `includes/class-gutenberg-sidebar.php:6457-6520`.
Impact:
- Multi-author sites can still leak or mutate post-scoped state across users.
- This is larger than conversations and should be fixed as a shared helper.
Recommended fix:
- Add a helper like `require_edit_post_or_error( $post_id, $action )`.
- Apply it to every route that reads or writes post meta, cost, image state, SEO/GEO state, or generated content.
## Medium Priority Findings
### P2: Provider Metadata Is Not Yet Consistent Across All AI Responses
Chat responses now include provider metadata, but planning/writing/refinement/clarity helper responses often only track cost internally and do not return provider/warning metadata to the frontend.
Evidence:
- Chat adds metadata in `includes/class-gutenberg-sidebar.php:1025-1029` and streaming complete event includes metadata in `includes/class-gutenberg-sidebar.php:1190-1199`.
- Many other calls use `$provider_result` internally but return older response shapes.
Opportunity:
- Standardize an AI response envelope for all REST/SSE flows: `content`, `provider`, `selected_provider`, `model`, `fallback_used`, `warnings`, `cost`, `usage`, `request_id`.
### P2: Model Defaults Are Still Fragmented
No single model registry exists yet. Defaults remain spread across activation, OpenRouter provider, settings V2, legacy settings, JS presets, Image Manager, and WP AI wrapper.
Evidence:
- Activation still stores `execution_model` in `wp-agentic-writer.php:140-142`.
- OpenRouter defaults differ across properties in `includes/class-openrouter-provider.php:34-69`.
- Settings V2 still has multiple fallback groups in `includes/class-settings-v2.php:105-111`, `includes/class-settings-v2.php:228-273`, and `includes/class-settings-v2.php:989-994`.
- JS presets remain in `assets/js/settings-v2.js:24-38`.
- Image Manager has its own writing/image fallbacks in `includes/class-image-manager.php:183-248`.
Recommendation:
- Add a PHP `Model_Registry` and localize it to settings JS.
- Stop writing `execution_model` for fresh installs.
### P2: Cost Table Schema Is Split Between Creation and Runtime ALTER
The runtime upgrader adds columns, but the base table creation in the main plugin file still creates the old schema.
Evidence:
- Base schema lacks provider/session/status in `wp-agentic-writer.php:198-209`.
- Runtime `ALTER TABLE` adds provider/session/status in `includes/class-cost-tracker.php:61-97`.
Risk:
- Fresh installs depend on Cost Tracker initialization to complete schema.
- Failed `DESCRIBE` on a missing table is not handled before `in_array()` checks.
Recommendation:
- Move the latest schema into `wp_agentic_writer_create_cost_table()`.
- Make runtime migrations table-existence safe and versioned.
### P2: Uninstall Is Still Incomplete And Duplicated
The main uninstall hook and `uninstall.php` remain inconsistent.
Evidence:
- Main uninstall deletes settings and some tables in `wp-agentic-writer.php:269-294`.
- `uninstall.php` deletes settings, `_wpaw_plan`, and cost table only in `uninstall.php:12-21`.
Still missing:
- `wpaw_conversations`, `wp_agentic_writer_custom_models`, DB version options, transients, scheduled `wpaw_cleanup_old_sessions`, `_wpaw_chat_history`, `_wpaw_memory`, `_wpaw_post_config`, writing-state meta, user preferences, and image post meta.
### P2: Debug Logging Remains Too Broad
There is a localized `debug` flag, but console logs and backend logs remain noisy.
Evidence:
- `wpAgenticWriter.debug` is localized in `includes/class-gutenberg-sidebar.php:257`.
- Frontend still logs migration/session/clarity details in `assets/js/sidebar.js:308-322`, `assets/js/sidebar.js:5619-5630`, and `assets/js/sidebar.js:6130-6157`.
- Settings JS logs model and cost debug info in `assets/js/settings-v2.js:52-130` and `assets/js/settings-v2.js:390-503`.
- Provider/backend `error_log()` calls remain in provider files and `class-gutenberg-sidebar.php`.
### P2: Changelog Policy Still Fails
`docs/DEFINITION_OF_DONE.md` requires updating `CHANGELOG.md`, but no `CHANGELOG.md` file was found.
## Recommended Next Work Queue
### Do Before Chat/Context Focus
1. Fix provider result contract callers in Image Manager, Keyword Suggester, and WP AI Client wrapper.
2. Restore streaming chat variable initialization.
3. Change Cost Tracker hook accepted args from 7 to 9 and add failure recording.
4. Fix conversation migration versioning and cleanup cron scheduling.
### Then Focus Chat/Context
1. Make `Context_Service::get_context()` migrate legacy history on read.
2. Delete or mark migrated `_wpaw_chat_history`.
3. Make clear-context clear both post context and active session messages.
4. Add a visible context inspector in the UI: active session, message count, linked post, focus keyword, language, provider/model, cost estimate.
### Later
1. Apply `edit_post` checks to every post-scoped route.
2. Centralize model defaults in a registry.
3. Consolidate uninstall/data retention.
4. Gate logging.
5. Add `CHANGELOG.md`.
## Conclusion
Do not switch solely to chat/context yet. The implementation is closer, but the provider contract mismatch and streaming-chat uninitialized state should be fixed first because they directly break runtime behavior and chat persistence. After those are resolved, chat/context is the right next focus.

199
docs/architecture/PLUGIN_AUDIT_RETRACE_EIGHTH_PASS_2026-05-25.md Normal file
View File

@@ -0,0 +1,199 @@
# WP Agentic Writer Eighth Retrace Audit
Status: COMPLETE / RETRACED
Completion marker: 2026-05-26
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_NINTH_PASS_2026-05-26.md`
Audit date: 2026-05-25
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_SEVENTH_PASS_2026-05-25.md`
Scope: eighth pass after seventh-retrace implementation, covering conversation history migration, provider transparency, cost tracking, model defaults, UI/UX, and release readiness.
## Executive Summary
The seventh-pass implementation made real progress:
- `WP_Agentic_Writer_Cost_Tracker::add_request()` now defaults unknown providers to `unknown` instead of `openrouter`.
- The previously sampled successful AI cost hooks now pass provider/session/status metadata.
- Previously missing provider metadata was added to more backend responses, including execution, regeneration, multi-pass refinement, and article refinement.
- The legacy `/chat-history` backend path now attempts to migrate/read conversation-backed data instead of always returning raw `_wpaw_chat_history`.
- PHP and JavaScript syntax checks pass.
However, the plugin is still not audit-clean. The most important remaining issue is conversation continuity: the sidebar still depends on the deprecated `/chat-history` route, and that route now appears internally inconsistent because it reads `_wpaw_active_session_id`, but no writer for that meta key was found. That can make legacy chat migration look successful in storage while the sidebar receives an empty history.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of seventh-pass findings against current code.
- Static sweep of chat-history migration, provider metadata, cost hook metadata, failed AI paths, sidebar provider UI, and model defaults.
- No live WordPress editor/browser workflow was run in this pass.
## Seventh-Pass Status Trace
| Seventh-pass item | Current status | Evidence |
|---|---:|---|
| Sidebar dependency on `/chat-history` | Still open | `assets/js/sidebar.js:644-668` still fetches `/chat-history/${postId}`. |
| Backend `/chat-history` compatibility | Partially fixed, has a new continuity bug | `get_post_chat_history()` migrates/reads sessions, but depends on `_wpaw_active_session_id` at `includes/class-gutenberg-sidebar.php:1359-1389`; no writer for that meta key was found. |
| Cost hook default provider | Fixed | `add_request()` now defaults provider to `unknown` at `includes/class-cost-tracker.php:120-124`. |
| Successful AI cost hook metadata | Improved | Previously sampled routes now pass provider/session/status, for example execution at `includes/class-gutenberg-sidebar.php:3269-3281`, regeneration at `includes/class-gutenberg-sidebar.php:3748-3760`, summarize at `includes/class-gutenberg-sidebar.php:6438-6450`, and article refinement at `includes/class-gutenberg-sidebar.php:7005-7017`. |
| Provider metadata on backend responses | Improved but inconsistent | More routes include `provider_metadata`, but chat still uses top-level fields and the sidebar does not render either shape. |
| Failed AI cost attempts | Still open | Several `is_wp_error()` branches still return without recording an error-status attempt. |
| Model registry/default unification | Still open | No central model registry found; defaults remain duplicated. |
| WordPress editor browser pass | Still open | Syntax checks passed, but no editor workflow was verified. |
## Remaining Findings
### P1: Deprecated Chat-History Compatibility Can Return Empty Data After Migration
The sidebar still loads chat history through the deprecated route:
- `assets/js/sidebar.js:644-668` fetches `${wpAgenticWriter.apiUrl}/chat-history/${postId}` and seeds `messages` from the response.
The backend route now tries to be smarter:
- `handle_get_chat_history()` calls `get_post_chat_history()` at `includes/class-gutenberg-sidebar.php:1300-1326`.
- `get_post_chat_history()` checks `_wpaw_chat_history_migrated`, then tries to read `_wpaw_active_session_id` and load context from that id at `includes/class-gutenberg-sidebar.php:1359-1368`.
- If legacy history exists and is not migrated, it calls `migrate_legacy_chat_history()` at `includes/class-gutenberg-sidebar.php:1378-1380`.
- After migration, it again tries `_wpaw_active_session_id` at `includes/class-gutenberg-sidebar.php:1382-1389`.
The problem: a search found no writer for `_wpaw_active_session_id`; it appears only in this read path. `migrate_legacy_chat_history()` returns the migrated session id at `includes/class-context-service.php:272-324`, but `get_post_chat_history()` ignores that return value.
Impact:
- A legacy post can be migrated into the conversation table and have `_wpaw_chat_history` deleted, but `/chat-history` can still return an empty array.
- Because the sidebar still depends on `/chat-history`, users can perceive this as "chat history disappeared" even though the session data exists.
- This is exactly the kind of context/history regression that keeps the audit loop alive.
Recommended fix:
- Best: stop using `/chat-history` in the sidebar and hydrate from the canonical conversation/session context endpoint.
- If the compatibility endpoint remains, capture the return value from `migrate_legacy_chat_history()` and load that session directly.
- For already migrated posts, use `WP_Agentic_Writer_Conversation_Manager::get_sessions_for_post( $post_id )` instead of `_wpaw_active_session_id`, or write `_wpaw_active_session_id` consistently when sessions are created/selected.
- Add a regression test for a legacy post with `_wpaw_chat_history` and no `_wpaw_active_session_id`; expected result: `/chat-history` returns migrated messages and the canonical session id.
### P1: Provider Transparency Is Still Not A Single End-To-End Contract
The backend now has a shared metadata helper:
- `build_provider_metadata()` returns `provider`, `selected_provider`, `fallback_used`, `warnings`, and `model` at `includes/class-gutenberg-sidebar.php:956-963`.
More routes now include provider metadata:
- Plan generation at `includes/class-gutenberg-sidebar.php:2049-2058`.
- Execution at `includes/class-gutenberg-sidebar.php:3283-3292`.
- Regeneration at `includes/class-gutenberg-sidebar.php:3762-3770`.
- Meta description at `includes/class-gutenberg-sidebar.php:6276-6285`.
- Multi-pass refinement at `includes/class-gutenberg-sidebar.php:6930-6939`.
- Article refinement at `includes/class-gutenberg-sidebar.php:7019-7028`.
But the contract is still not end to end:
- Chat still returns top-level `provider`, `selected_provider`, `fallback_used`, and `warnings` at `includes/class-gutenberg-sidebar.php:1067-1071`, while other routes use nested `provider_metadata`.
- `assets/js/sidebar.js` does not reference `provider_metadata`, `fallback_used`, `selected_provider`, or `warnings`; its only provider references are web-search availability checks at `assets/js/sidebar.js:5983-6000`.
Impact:
- API consumers still need two response shapes.
- Users still do not see actual provider/fallback behavior in the editor.
- The Definition of Done says UI must show actual provider used, but that is still not implemented.
Recommended fix:
- Choose one response shape and apply it to all AI endpoints. If keeping `provider_metadata`, also make chat use that envelope.
- Update sidebar state/rendering to show actual provider, model, fallback, and warnings near cost/status feedback.
- Add a static check that every provider-backed route returns the same metadata shape.
### P2: Failed AI Calls Still Usually Do Not Record Error-Status Cost Attempts
Successful cost tracking improved substantially, but failed attempts are still mostly invisible:
- Clarity fallback returns cost `0` without recording a failed provider attempt at `includes/class-gutenberg-sidebar.php:4100-4110`.
- Clarity JSON parse fallback returns cost `0` without an error-status record at `includes/class-gutenberg-sidebar.php:4117-4132`.
- Regeneration returns `regeneration_error` without cost/error tracking at `includes/class-gutenberg-sidebar.php:3738-3746`.
- Multi-pass refinement returns the provider error directly at `includes/class-gutenberg-sidebar.php:6910-6914`.
- Article refinement returns the provider error directly at `includes/class-gutenberg-sidebar.php:6996-7000`.
Impact:
- Provider reliability and failure rates are undercounted.
- Users may see a failure, but admins do not get a durable cost/attempt trail explaining which provider/model failed.
- Fallback behavior is harder to audit because failures and successes are not represented consistently.
Recommended fix:
- Add one helper, for example `track_ai_cost( $post_id, $response, $action, $provider_result, $session_id = '', $status = 'success' )`.
- Call it for both success and failure paths, with `status = 'error'`, `cost = 0`, and available provider/model data for failures.
- Keep user-facing errors unchanged, but always record the attempt where a provider request was actually made.
### P2: Cost Tracking Is Better But Still Not Structurally Guarded
The direct default-provider bug was fixed:
- `add_request()` now defaults provider to `unknown` at `includes/class-cost-tracker.php:124`.
The previous high-risk success hooks sampled in this pass now include provider/session/status. That is good. The remaining structural issue is that cost tracking is still done through repeated raw `do_action( 'wp_aw_after_api_request', ... )` calls across the route class.
Impact:
- New AI routes can still accidentally omit provider/session/status, or skip failed-attempt tracking.
- The system relies on manual discipline instead of a reusable contract.
Recommended fix:
- Wrap the hook with a local helper and use that helper everywhere.
- Add a simple static test or lint script that rejects direct `do_action( 'wp_aw_after_api_request'` outside the helper.
### P2: Model Defaults Are Still Fragmented
No central model registry was found. Defaults remain spread across:
- Activation defaults in `wp-agentic-writer.php:140-142`.
- Sidebar defaults in `includes/class-gutenberg-sidebar.php:278-283`.
- Settings defaults and fallbacks in `includes/class-settings.php` and `includes/class-settings-v2.php`.
- OpenRouter provider defaults in `includes/class-openrouter-provider.php:34-69`.
- JavaScript presets in `assets/js/settings-v2.js:35-56`.
- Wrapper fallback model groups in `includes/class-wp-ai-client-wrapper.php:94-100`.
- Image manager fallbacks in `includes/class-image-manager.php:185-249`.
Impact:
- Defaults can drift between activation, runtime provider behavior, settings UI, cost estimation, and JS presets.
- Model-related fixes remain easy to regress.
Recommended fix:
- Create one PHP model registry for task defaults, labels, capabilities, provider support, pricing hints, and deprecation status.
- Localize JS presets from that registry.
- Add a consistency check that activation/settings/provider defaults match the registry.
### P2: Editor UI/UX Still Needs Browser Verification
Syntax checks are clean, but no live editor workflow was run.
Impact:
- The remaining issues are heavily UI-dependent: chat hydration, session continuity, provider warning display, streaming completion, and cost feedback.
- Static checks cannot validate editor package compatibility, REST nonce behavior, layout, or user-visible provider/cost states.
Recommended fix:
- Run a WordPress editor browser pass after the next implementation pass.
- Verify sidebar open/persist, chat reload continuity, plan/write/refine cost updates, provider warning display, and unauthorized post failures.
## Recommended Next Work
1. Remove the sidebar dependency on `/chat-history`, or repair the compatibility route by using the migrated session id directly.
2. Add frontend rendering for provider/model/fallback/warnings.
3. Normalize provider metadata to one response shape across chat and non-chat AI actions.
4. Add failed-attempt cost tracking with `status = 'error'`.
5. Wrap cost tracking in a helper and stop using raw hook calls directly.
6. Consolidate model defaults into a registry.
7. Run the WordPress editor browser workflow pass.
## Current Verdict
The seventh-pass implementation is partially proper and materially better than the previous state. Successful cost attribution and backend provider metadata coverage improved.
The plugin is still not audit-clean. The highest-priority remaining fix is chat/context continuity: the active sidebar history loader still depends on a deprecated route, and that route can return empty results after migration because it relies on `_wpaw_active_session_id` without any discovered writer.

169
docs/architecture/PLUGIN_AUDIT_RETRACE_ELEVENTH_PASS_2026-05-26.md Normal file
View File

@@ -0,0 +1,169 @@
# WP Agentic Writer Eleventh Retrace Audit
Audit date: 2026-05-26
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_TENTH_PASS_2026-05-26.md`
Scope: eleventh pass after tenth-retrace implementation, covering provider transparency coverage, model registry adoption, chat/context compatibility, cost tracking contracts, UI/UX, and release readiness.
Status: COMPLETE / RETRACED
Completion marker: 2026-05-26
Follow-up retrace: `docs/architecture/PLUGIN_AUDIT_RETRACE_TWELFTH_PASS_2026-05-26.md`
> This eleventh-pass report has been implemented and retraced. Keep this document as historical evidence only; use the twelfth-pass report for current remaining work.
## Executive Summary
The tenth-pass implementation improved the plugin again:
- A shared frontend `applyProviderMetadata()` helper now exists.
- Several AI response paths call that helper, so provider/fallback metadata reaches the UI in more than just the original stream completion path.
- The P0 failed-attempt fatal from the ninth pass remains fixed.
- PHP and JavaScript syntax checks pass.
No new P0 blocker was found. The remaining issues are now narrower, but still real:
- Provider metadata UI coverage is broader, but still not complete across all AI response paths.
- Model registry adoption is improved, but some active and fallback paths still carry hard-coded model IDs.
- The sidebar still hydrates chat through deprecated `/chat-history`.
- Raw cost hook calls still bypass the `track_ai_cost()` helper in many places.
- Live WordPress editor browser verification is still pending.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of tenth-pass findings against current code.
- Static sweep of provider metadata UI usage, model registry adoption, chat-history usage, and raw cost hooks.
- No live WordPress editor/browser workflow was run in this pass.
## Tenth-Pass Status Trace
| Tenth-pass item | Current status | Evidence |
|---|---:|---|
| Shared frontend provider metadata helper | Fixed | `applyProviderMetadata()` exists at `assets/js/sidebar.js:75-91`. |
| Provider metadata UI coverage | Improved, still partial | Helper is called at `assets/js/sidebar.js:1043`, `1772`, `1995`, `2281`, and `3349`, but some AI response paths still do not call it. |
| Provider badge rendering | Fixed for covered paths | Provider/fallback badge renders near cost at `assets/js/sidebar.js:4677-4720`. |
| Model registry adoption | Improved, still partial | Active settings/sidebar paths use `WPAW_Model_Registry`, but fallback model lists, JS presets, provider property defaults, and some image paths still hard-code model IDs. |
| Sidebar `/chat-history` dependency | Still open | `assets/js/sidebar.js:666-680` still fetches `/chat-history/${postId}`. |
| `/chat-history` docblock mismatch | Still open | Docblock still says the endpoint does not use the conversations table at `includes/class-gutenberg-sidebar.php:1337-1339`. |
| Raw cost hook drift | Still open | Direct `do_action( 'wp_aw_after_api_request', ... )` calls remain in `includes/class-gutenberg-sidebar.php` outside `track_ai_cost()`. |
| Browser verification | Still open | Syntax checks passed, but no live editor workflow was verified. |
## Remaining Findings
### P1: Provider Metadata UI Coverage Is Still Partial
The new frontend helper is good:
- `applyProviderMetadata()` supports both `provider_metadata` and top-level provider fields at `assets/js/sidebar.js:75-91`.
- It is called from several important paths, including streaming completion and some JSON responses at `assets/js/sidebar.js:1043`, `1772`, `1995`, `2281`, and `3349`.
- The provider/fallback badge is rendered at `assets/js/sidebar.js:4677-4720`.
However, not every AI response path applies provider metadata yet. Examples:
- Meta generation parses JSON at `assets/js/sidebar.js:595-605` but does not call `applyProviderMetadata()`.
- Summarize context parses JSON at `assets/js/sidebar.js:1601-1612` but does not call it.
- Intent detection parses JSON at `assets/js/sidebar.js:1644-1648` but does not call it.
- Reformat blocks parses JSON at `assets/js/sidebar.js:2191-2219` but does not call it.
- Refine-from-chat streaming parses data events at `assets/js/sidebar.js:2771-2828` but does not apply metadata on completion.
Impact:
- The provider badge can be stale after some AI actions.
- Users may see provider information for generation/chat but not for meta, summarization, intent, reformat, or refinement workflows.
- This still falls short of a consistent provider transparency contract.
Recommended fix:
- Call `applyProviderMetadata(data)` immediately after every AI JSON response parse.
- Call it on every streaming `complete` event, including refine-from-chat.
- Add a quick static check that every fetch to an AI endpoint either calls `applyProviderMetadata()` or explicitly comments why provider metadata is not expected.
### P1: Model Registry Still Is Not The Sole Source Of Truth
Registry adoption improved in active settings and sidebar defaults, but hard-coded model IDs remain in several places:
- `includes/class-settings-v2.php:188-215` still contains fallback model arrays with literal model IDs.
- `includes/class-settings-v2.php:224-230` still uses literal fallback IDs in model transformation.
- `assets/js/settings-v2.js:32-58` still hard-codes budget/balanced/premium preset IDs.
- `includes/class-openrouter-provider.php:29-75` still hard-codes provider property defaults, and the constructor uses those properties when settings are absent at `includes/class-openrouter-provider.php:437-448`.
- `includes/class-image-manager.php:409-478` still hard-codes image model fallbacks.
- Legacy `includes/class-settings.php` still contains hard-coded defaults and may be instantiated if Settings V2 is unavailable.
Impact:
- The registry can still drift from runtime behavior.
- The settings UI fallback list can disagree with generation defaults.
- Future model changes still require touching multiple locations.
Recommended fix:
- Replace remaining active runtime fallbacks with `WPAW_Model_Registry::get_default_model()` or `get_fallback_model()`.
- Treat JS presets as curated presets and document them as such, or generate them from localized registry data.
- Initialize OpenRouter provider defaults from the registry in the constructor.
- Replace image manager fallback literals at `includes/class-image-manager.php:409-478`.
- Decide whether legacy `class-settings.php` is supported; if yes, update its defaults to use the registry, otherwise remove fallback instantiation.
### P2: Sidebar Still Uses Deprecated `/chat-history`
The route compatibility bug from earlier passes appears fixed, but the sidebar still uses the deprecated route:
- `assets/js/sidebar.js:666-680` fetches `/chat-history/${postId}`.
- The backend route remains registered at `includes/class-gutenberg-sidebar.php:346-354`.
- The docblock still says the endpoint does not use conversations at `includes/class-gutenberg-sidebar.php:1337-1339`, even though the implementation now reads session-backed history.
Impact:
- The UI still depends on a compatibility endpoint.
- Documentation and behavior disagree.
- Future cleanup can break chat hydration again.
Recommended fix:
- Move sidebar hydration to the canonical conversation/session context endpoint.
- If `/chat-history` remains, update the docblock and response contract to explicitly say it returns session-backed compatibility data.
### P2: Cost Tracking Helper Is Not Yet Enforced
`track_ai_cost()` exists, but raw cost hook calls remain:
- `includes/class-gutenberg-sidebar.php` still has many direct `do_action( 'wp_aw_after_api_request', ... )` calls outside the helper.
- `includes/class-keyword-suggester.php:122` also calls the cost hook directly.
Impact:
- New changes can still bypass provider/session/status normalization.
- The codebase still relies on manual discipline instead of enforcing the cost tracking contract.
Recommended fix:
- Convert remaining route-level raw hooks to `track_ai_cost()`.
- Either expose a shared cost helper outside the sidebar class or document why non-sidebar callers may use the raw hook.
- Add a static guard that only allows raw `wp_aw_after_api_request` calls in approved files/lines.
### P2: Live Editor Browser Verification Still Remains
No live WordPress editor browser workflow was run in this retrace.
Recommended browser checklist:
- Sidebar opens and persists in the block editor.
- Chat session continues after page reload.
- Provider/fallback warnings render after every AI action with metadata.
- Cost display updates after chat, plan, refine, and meta actions.
- Unauthorized post access fails cleanly.
- Model settings changes reflect in generated requests.
## Recommended Next Work
1. Add `applyProviderMetadata()` to the remaining AI response paths.
2. Finish model registry adoption in active runtime paths or explicitly document curated exceptions.
3. Move sidebar chat hydration off `/chat-history`, or update the route contract/docblock.
4. Convert remaining raw cost hooks or add a static guard for approved direct hook use.
5. Run the live WordPress editor browser workflow pass.
## Current Verdict
The tenth-pass implementation is proper for the provider-UI helper it targeted and does not introduce a new P0. The audit chain is now mostly down to consistency, cleanup, and browser validation.
I would not call the plugin fully audit-clean yet, but the remaining issues are bounded and should be much smaller to close than the earlier authorization/context/cost blockers.

127
docs/architecture/PLUGIN_AUDIT_RETRACE_FIFTEENTH_PASS_2026-05-26.md Normal file
View File

@@ -0,0 +1,127 @@
# WP Agentic Writer Fifteenth Retrace Audit
Audit date: 2026-05-26
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_FOURTEENTH_PASS_2026-05-26.md`
Scope: fifteenth pass after fourteenth-retrace implementation, covering retry-chat provider metadata, live editor readiness, model preset ownership, syntax verification, and remaining audit-chain debt.
Status: COMPLETE / RETRACED
Completion marker: 2026-05-26
Follow-up retrace: `docs/architecture/PLUGIN_AUDIT_RETRACE_SIXTEENTH_PASS_2026-05-26.md`
> This fifteenth-pass report has been implemented and retraced. Keep this document as historical evidence only; use the sixteenth-pass report for current remaining work.
## Executive Summary
The fourteenth-pass implementation closed the last concrete provider metadata gap from the previous report:
- Retry chat stream completion now calls `applyProviderMetadata(data)` at `assets/js/sidebar.js:1186-1189`.
- The full-contract cost hook state remains clean: static scan finds only the central helper hook and the keyword suggester full-contract hook.
- The legacy chat migration P0 remains fixed: no direct `new WP_Agentic_Writer_Context_Service` references were found.
- PHP and JavaScript syntax checks pass.
No new P0 or P1 blocker was found.
At this point, the audit chain is no longer finding major static implementation defects in the chat/context/provider/cost/model paths. The main remaining release gate is live WordPress editor/browser verification. Two small cleanup opportunities remain: duplicated model preset fallback/legacy maps, and a duplicate frontend `applyProviderMetadata()` call in one normal generation branch.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of fourteenth-pass findings against current code.
- Static scan for short-form `wp_aw_after_api_request` calls.
- Static scan for direct `new WP_Agentic_Writer_Context_Service`.
- Static scan for provider metadata completion branches.
- Static scan for live browser verification evidence.
- No live WordPress editor/browser workflow was run in this pass.
## Fourteenth-Pass Status Trace
| Fourteenth-pass item | Current status | Evidence |
|---|---:|---|
| Retry chat applies provider metadata | Fixed | `assets/js/sidebar.js:1186-1189` calls `applyProviderMetadata(data)` on retry-chat completion. |
| Live editor/browser verification | Still open | No new browser verification note or evidence was found. |
| Curated preset duplication | Improved, still partial | Settings V2 now localizes `get_model_presets()` from PHP, but JS fallback and legacy settings still duplicate preset maps. |
## Remaining Findings
### P2: Live WordPress Editor Browser Verification Is Now The Main Gate
Static checks are clean enough that the next confidence jump needs a live editor pass.
Required browser verification:
- Legacy `_wpaw_chat_history` migrates through `/conversation/{post_id}` without fatal error.
- Sidebar chat persists after editor reload.
- Retry chat updates the provider/fallback badge.
- Provider badge updates after chat, clarity, planning, generation, block refinement, chat refinement, meta, keyword, intent, and improvement actions.
- Cost log rows include provider/session/status for the same actions.
- Model setting changes affect generated requests.
- Unauthorized REST access remains denied.
Impact:
- Without this pass, the audit chain can prove static contract cleanup, but not editor UI behavior, persistence, REST permission behavior, or visual state updates inside WordPress.
Recommended fix:
- Run the plugin in a live WordPress editor and save a short verification note with exact workflows checked, post IDs used, and any screenshots/log notes.
- If automated browser coverage is possible, capture at least sidebar load/reload, legacy migration, provider badge change, and cost log attribution.
### P3: Curated Model Presets Are Centralized For Settings V2, But Fallback/Legacy Duplicates Remain
Settings V2 now has a PHP source for curated presets:
- `includes/class-settings-v2.php:136-162` defines `get_model_presets()`.
- `includes/class-settings-v2.php:100-113` localizes those presets into `wpawSettingsV2`.
- `assets/js/settings-v2.js:32-35` uses `wpawSettingsV2?.presets`.
Remaining duplication:
- `assets/js/settings-v2.js:35-60` still contains a hard-coded fallback preset map if localization is missing.
- `includes/class-settings.php:1025-1055` still contains a legacy inline preset map.
- `wp-agentic-writer.php:100-104` can still instantiate the legacy settings class when Settings V2 is not selected.
Impact:
- This is no longer a high-risk active Settings V2 defect, but preset updates can still drift across fallback/legacy code.
Recommended fix:
- For Settings V2, either remove the hard-coded JS fallback or make it an empty/no-op fallback with an admin notice if localization is missing.
- For legacy settings, either read the V2 preset source or formally mark legacy preset parity as manually maintained.
### P3: Duplicate Provider Metadata Call In Normal Generation Branch
One stream completion branch now calls `applyProviderMetadata(data)` twice:
- `assets/js/sidebar.js:1039-1045` calls it before and after cost update.
Impact:
- This is harmless, but it creates audit noise and unnecessary React state churn.
Recommended fix:
- Keep one call in that completion branch.
## Closed In This Pass
- Retry-chat provider metadata application is fixed.
- No direct context-service construction was found.
- No short-form cost hook calls were found.
- Syntax checks passed for PHP and key JavaScript files.
## Priority Queue
1. P2: Run live WordPress editor/browser verification and record evidence.
2. P3: Decide ownership for JS fallback and legacy model preset duplication.
3. P3: Remove duplicate `applyProviderMetadata(data)` call in the normal generation completion branch.
## Completion Criteria For Next Pass
The next retrace can mark this pass complete when:
- Live editor verification evidence exists for migration, persistence, provider badge updates, cost attribution, model settings, retry chat, and auth denial.
- Preset duplication is centralized or explicitly accepted as manually maintained legacy/fallback behavior.
- The duplicate provider metadata call is removed or intentionally left with a comment.

177
docs/architecture/PLUGIN_AUDIT_RETRACE_FIFTH_PASS_2026-05-25.md Normal file
View File

@@ -0,0 +1,177 @@
# WP Agentic Writer Fifth Retrace Audit
Status: COMPLETE / RETRACED
Completion marker: 2026-05-25
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_SIXTH_PASS_2026-05-25.md`
Audit date: 2026-05-25
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_FOURTH_PASS_2026-05-25.md`
Scope: fifth pass after fourth-retrace implementation, covering REST authorization, conversation context/history, provider metadata, cost attribution, model defaults, and release readiness.
## Executive Summary
The fourth-pass implementation closed meaningful gaps:
- `handle_clear_context()` now checks `edit_post` before clearing post context.
- Legacy chat migration now deletes `_wpaw_chat_history` and writes `_wpaw_chat_history_migrated`.
- `get_context()` now attempts migrate-on-read when no session exists and legacy post meta is present.
- `record_usage()` is now marked deprecated.
- PHP syntax validation still passes.
- `assets/js/sidebar.js` and `assets/js/settings-v2.js` still pass JavaScript syntax validation.
The plugin is closer, but not fully clean. The remaining problem is no longer broad absence of checks; it is **ordering and coverage**. Several handlers added permission checks, but some still read post config/meta before checking access. Other AI utility routes accept `postId` for cost attribution or context but still never verify that the current user can edit that post.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- Static trace of fourth-pass findings against current code.
- No live WordPress browser workflow was run in this pass.
## Fourth-Pass Status Trace
| Fourth-pass item | Current status | Evidence |
|---|---:|---|
| Clear-context post auth | Fixed | `handle_clear_context()` checks `check_post_permission()` before clearing at `includes/class-gutenberg-sidebar.php:1240-1263`. |
| Legacy chat migration cleanup | Fixed | `migrate_legacy_chat_history()` deletes legacy meta and writes `_wpaw_chat_history_migrated` at `includes/class-context-service.php:310-312`. |
| Migrate-on-read behavior | Improved | `get_context()` triggers migration when no session exists and legacy history is present at `includes/class-context-service.php:62-78`. |
| Deprecated legacy cost method | Improved | `record_usage()` is marked deprecated at `includes/class-cost-tracker.php:162-188`. |
| Permission sweep | Partially fixed | Several routes now check, but some checks still happen after post reads and some post-id routes still lack checks. |
| Provider metadata response contract | Still open | Chat exposes provider metadata; non-chat generated responses remain inconsistent. |
| Model registry/default unification | Still open | Defaults remain spread across PHP, JS, providers, wrapper, and image manager. |
## Critical Findings
### P0: Some Permission Checks Still Happen After Post-Scoped Reads
The fourth-pass report asked for checks before any post read/write/cost attribution/streaming. Some handlers added checks, but they are still too late:
- `handle_revise_plan()` calls `resolve_post_config_from_request()` and reads `_wpaw_detected_language` before checking `edit_post` at `includes/class-gutenberg-sidebar.php:2023-2057`.
- `handle_block_refine()` calls `resolve_post_config_from_request()` before checking `edit_post` at `includes/class-gutenberg-sidebar.php:4203-4230`.
- `handle_refine_from_chat()` calls `resolve_post_config_from_request()` before checking `edit_post` at `includes/class-gutenberg-sidebar.php:4857-4885`.
- `handle_generate_meta()` reads post content/title with `get_post()` before checking `edit_post` at `includes/class-gutenberg-sidebar.php:6077-6108`.
- `handle_check_clarity()` calls `resolve_post_config_from_request()` before checking `edit_post` at `includes/class-gutenberg-sidebar.php:3809-3839`.
Impact:
- A user can still trigger reads of post config, detected language, post title/content, or other post-scoped metadata for posts they cannot edit.
- These are not merely theoretical because helper calls like `resolve_post_config_from_request()` fall back to stored post config.
Recommended fix:
- Move the post permission check immediately after extracting `postId`, before any helper call that may read post meta/content.
- Use one centralized helper so the ordering is hard to get wrong:
- Extract post id.
- If `postId > 0`, require `edit_post`.
- Only then read config/meta/content or start streaming.
### P0: Some Post-ID Utility Routes Still Have No Target-Post Check
Several routes still accept `postId` and use it for context or cost attribution without validating target-post access:
- `handle_summarize_context()` accepts `postId` and records cost against it without checking `edit_post` at `includes/class-gutenberg-sidebar.php:6258-6330`.
- `handle_detect_intent()` accepts `postId` and records cost against it without checking `edit_post` at `includes/class-gutenberg-sidebar.php:6357-6415`.
- `handle_refine_multi_pass()` accepts `postId` and records cost against it without checking `edit_post` at `includes/class-gutenberg-sidebar.php:6730-6782`.
Impact:
- Cost records can still be attributed to posts the user cannot edit.
- These endpoints can be used to pollute another post's cost ledger even if they do not read that post's content directly.
Recommended fix:
- If an endpoint accepts `postId > 0`, require `edit_post` before using it for cost tracking.
- If the endpoint does not need post authority, ignore client-provided `postId` and track cost against `0` or the active session instead.
## High Priority Findings
### P1: Provider Metadata Is Still Not Uniform Outside Chat
Provider selection metadata exists in many code paths, but only chat responses consistently expose it. Non-chat flows still often return content, blocks, variants, keyword suggestions, or SEO results without a consistent provider envelope.
Examples from the current trace:
- Plan/revise/execute/refine handlers use `WP_Agentic_Writer_Provider_Manager::get_provider_for_task()`, but response envelopes do not consistently return `provider`, `selected_provider`, `fallback_used`, `warnings`, `model`, and `cost`.
- Keyword and image helper classes unwrap provider results but do not expose fallback metadata to the calling UI consistently.
Impact:
- Users and support logs still cannot reliably answer "which provider/model served this request?" outside chat.
- Fallback behavior is harder to debug in plan, refinement, SEO/GEO, keyword, and image paths.
Recommended fix:
- Add a shared helper that builds provider metadata from `WPAW_Provider_Selection_Result` plus model/cost response data.
- Add it to every generated response and streaming completion event.
### P1: Migrate-On-Read May Not Return the Newly Created Session
`get_context()` now calls `migrate_legacy_chat_history( $post_id, $session_id )`, but `migrate_legacy_chat_history()` only accepts `$post_id` in its signature and creates a new session when none exists. After that, `get_context()` tries `$manager->get_session( $session_id )` again.
Evidence:
- `get_context()` calls migration with two arguments at `includes/class-context-service.php:71-73`.
- `migrate_legacy_chat_history()` signature is `migrate_legacy_chat_history( $post_id )` at `includes/class-context-service.php:267`.
- When no session exists, migration creates a new session but does not return that new session id at `includes/class-context-service.php:301-307`.
Impact:
- Legacy history can be migrated and deleted, but the same read may still return an empty context if the caller's original `$session_id` was not the newly created session.
- The data is safer than before, but UX may still appear as "history disappeared" until a separate session lookup/list refresh.
Recommended fix:
- Make `migrate_legacy_chat_history()` return the target/new `session_id`, or accept a requested session id and create/update that session.
- In `get_context()`, fetch the returned session id after migration.
## Medium Priority Findings
### P2: Legacy `/chat-history` Endpoint Still Exposes Deprecated Storage
The route now has permission checks and a deprecated response marker, and legacy writes are disabled. Still, `/chat-history/(?P<post_id>\d+)` remains active and reads `_wpaw_chat_history`.
Impact:
- A deprecated route can keep old UI/client assumptions alive.
- It can confuse the source-of-truth contract if any caller still consumes it.
Recommended fix:
- Return an empty deprecated response after the migration window, or remove the route once the sidebar is fully on conversation sessions.
### P2: Legacy `record_usage()` Still Defaults Provider to OpenRouter
The method is deprecated, but it still records provider as `openrouter` for any caller that uses it.
Recommended fix:
- Change the fallback provider to `unknown`, or require callers to migrate to `record_usage_full()` before accepting records from this compatibility path.
### P2: Model Defaults Remain Fragmented
Model defaults still live across activation, settings PHP, settings JS, providers, image manager, and WP AI wrapper. This is unchanged from the fourth pass.
Recommended fix:
- Add a PHP model registry/default resolver.
- Localize resolved defaults to JS.
- Treat legacy `execution_model` as a migration alias into canonical `writing_model`.
### P2: Sidebar WordPress Compatibility Still Needs Browser Verification
The sidebar still imports `PluginSidebar` from `wp.editPost`. The previous fallback was removed. Syntax is green, but compatibility still needs a browser check on the minimum supported WordPress version.
## Definition of Done Gates for This Pass
Before considering this pass complete:
- Move every post permission check before any post config/meta/content read.
- Add target-post checks or ignore `postId` in summarize-context, detect-intent, and refine-multi-pass.
- Make legacy migrate-on-read return the migrated session in the same request.
- Standardize provider metadata in non-chat generated responses.
- Keep PHP and JS syntax checks green.
## Current Decision
The fourth-pass implementation is a real improvement and the legacy history storage is much closer to sane. Do not shift fully into new chat/context work yet. First finish the permission-order sweep and the few remaining cost-attribution endpoints; then the remaining issues are mostly consistency and observability rather than core safety.

127
docs/architecture/PLUGIN_AUDIT_RETRACE_FOURTEENTH_PASS_2026-05-26.md Normal file
View File

@@ -0,0 +1,127 @@
# WP Agentic Writer Fourteenth Retrace Audit
Audit date: 2026-05-26
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_THIRTEENTH_PASS_2026-05-26.md`
Scope: fourteenth pass after thirteenth-retrace implementation, covering legacy chat migration, cost attribution, provider metadata propagation, model preset ownership, UI/UX readiness, and release verification.
Status: COMPLETE / RETRACED
Completion marker: 2026-05-26
Follow-up retrace: `docs/architecture/PLUGIN_AUDIT_RETRACE_FIFTEENTH_PASS_2026-05-26.md`
> This fourteenth-pass report has been implemented and retraced. Keep this document as historical evidence only; use the fifteenth-pass report for current remaining work.
## Executive Summary
The thirteenth-pass implementation closed the serious issues from the previous report:
- The P0 legacy migration fatal is fixed statically. The canonical `/conversation/{post_id}` migration path now uses `WP_Agentic_Writer_Context_Service::get_instance()` instead of direct construction.
- The broad seven-argument cost hook drift is fixed statically. A scan now finds only the central helper hook and the keyword suggester full-contract hook.
- The previously listed stream completion payloads now include provider metadata where provider-backed output was involved.
- The frontend stream completion branch at `assets/js/sidebar.js:4223-4226` now applies provider metadata.
No new P0 or P1 blocker was found in this pass.
The remaining work is narrower:
- One retry-chat streaming completion path still does not apply provider metadata even though `/chat` completion sends provider fields.
- Live WordPress editor/browser verification is still not evidenced.
- Curated model presets remain duplicated between Settings V2 JavaScript and the legacy settings UI.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of thirteenth-pass findings against current code.
- Static scan for short-form `wp_aw_after_api_request` calls.
- Static scan for direct `new WP_Agentic_Writer_Context_Service`.
- Static scan for provider metadata stream completion gaps.
- No live WordPress editor/browser workflow was run in this pass.
## Thirteenth-Pass Status Trace
| Thirteenth-pass item | Current status | Evidence |
|---|---:|---|
| P0 direct context-service construction | Fixed | `includes/class-gutenberg-sidebar.php:1413` uses `WP_Agentic_Writer_Context_Service::get_instance()`. |
| Seven-argument cost hooks | Fixed statically | Hook scan finds `includes/class-gutenberg-sidebar.php:1004` central helper and `includes/class-keyword-suggester.php:140` full-contract hook only. |
| Stream completion metadata payloads | Mostly fixed | Provider metadata now exists at `includes/class-gutenberg-sidebar.php:3730-3733`, `4831-4834`, and `5567-5570`. |
| Frontend stream completion metadata application | Mostly fixed | `assets/js/sidebar.js:4223-4226` now calls `applyProviderMetadata(data)`. |
| Browser verification | Still open | Static checks passed; no live editor workflow evidence was found. |
| Curated model preset duplication | Still open, low priority | Presets remain duplicated in `assets/js/settings-v2.js` and `includes/class-settings.php`. |
## Remaining Findings
### P2: Retry Chat Completion Does Not Apply Provider Metadata
Most stream completion branches now call `applyProviderMetadata(data)`, but the retry-chat path still does not.
Evidence:
- Retry chat posts to `/chat` with `stream: true` at `assets/js/sidebar.js:1140-1152`.
- Its completion branch at `assets/js/sidebar.js:1186-1200` finalizes the streaming assistant message and extracts keyword suggestions, but does not call `applyProviderMetadata(data)`.
- The backend chat stream completion sends provider transparency fields at `includes/class-gutenberg-sidebar.php:1290-1298`.
Impact:
- After retrying a failed chat request, the provider/fallback badge can remain stale even though the completion event has provider data.
- The normal chat path and many generation/refinement paths are covered, so this is now a focused UI consistency gap rather than a systemic provider transparency failure.
Recommended fix:
- Add `applyProviderMetadata(data)` inside the retry-chat `data.type === 'complete'` branch before or after the message finalization.
- Optionally remove the duplicate `applyProviderMetadata(data)` call in the normal generation branch at `assets/js/sidebar.js:1039-1045` while touching the area.
### P2: Live Browser Verification Is Still Required
Static checks are clean, but the audit chain still has no live editor evidence.
The browser pass should verify:
- Legacy `_wpaw_chat_history` migrates through `/conversation/{post_id}` without fatal error.
- Sidebar chat persists after editor reload.
- Retry chat updates provider/fallback badge.
- Provider badge updates after chat, clarity, planning, generation, block refinement, chat refinement, meta, keyword, intent, and improvement actions.
- Cost log rows include provider/session/status for the same actions.
- Model setting changes affect generated requests.
- Unauthorized REST access remains denied.
### P3: Curated Model Presets Remain Duplicated
This is now a low-priority maintenance issue, not a release blocker.
Evidence:
- Active Settings V2 presets remain hard-coded at `assets/js/settings-v2.js:35-59`.
- Legacy settings presets remain hard-coded at `includes/class-settings.php:1027-1051`.
- The legacy settings class can still be instantiated at `wp-agentic-writer.php:100-104`.
Impact:
- Presets can drift between the active and legacy settings UIs.
- Model updates still require edits in more than one preset location.
Recommended fix:
- Keep curated presets, but centralize them in one PHP source and localize them into both UIs.
- If the legacy settings UI is truly fallback-only, document who owns preset parity.
## Closed In This Pass
- No direct `new WP_Agentic_Writer_Context_Service` references remain.
- No short-form seven-argument `wp_aw_after_api_request` calls remain in `class-gutenberg-sidebar.php`.
- The previously missing stream completion metadata payloads are now present for provider-backed refinement/generation paths.
- JavaScript and PHP syntax checks pass.
## Priority Queue
1. P2: Add provider metadata application to retry-chat stream completion.
2. P2: Run live WordPress editor/browser verification.
3. P3: Centralize or formally own curated preset duplication.
## Completion Criteria For Next Pass
The next retrace can mark this pass complete when:
- Retry chat applies provider metadata on stream completion.
- Live editor verification evidence exists for legacy migration, chat persistence, provider badge updates, cost attribution, model settings, and auth denial.
- Any remaining duplicated model presets are either centralized or intentionally owned as product presets.

181
docs/architecture/PLUGIN_AUDIT_RETRACE_FOURTH_PASS_2026-05-25.md Normal file
View File

@@ -0,0 +1,181 @@
# WP Agentic Writer Fourth Retrace Audit
Status: COMPLETE / SUPERSEDED
Completion marker date: 2026-05-25
Next retrace report: `docs/architecture/PLUGIN_AUDIT_RETRACE_FIFTH_PASS_2026-05-25.md`
Audit date: 2026-05-25
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_THIRD_PASS_2026-05-24.md`
Scope: fourth pass after third-retrace implementation, covering UI/UX runtime, REST authorization, conversation context/history, cost tracking, provider/model metadata, migrations, and release readiness.
## Executive Summary
The third-pass implementation closed several important items:
- Sidebar logger recursion is fixed. `wpawLog` now calls `console.*` directly.
- PHP syntax validation passes.
- `assets/js/sidebar.js` and `assets/js/settings-v2.js` pass JavaScript syntax validation.
- Cost table runtime upgrade now checks for a missing table and recreates it.
- WP AI wrapper cost tracking now calls `record_usage_full()` with provider/model metadata instead of the legacy incomplete method.
- Chat, generate-plan, execute-article, reformat-block, and regenerate-block handlers received target-post permission checks.
- Clear-context now clears all active sessions for a post when the frontend does not send a `sessionId`.
- Legacy `update_post_chat_history()` no longer writes `_wpaw_chat_history`.
The remaining blockers are now concentrated in authorization and source-of-truth cleanup. The biggest issue is that several REST handlers still accept `postId` and read or mutate post-specific data before checking `edit_post`, or without checking it at all. The second issue is that legacy chat-history migration remains manual and keeps migrated meta in place.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- Static trace of third-pass findings against current code.
- No live WordPress browser workflow was run in this pass.
## Third-Pass Status Trace
| Third-pass item | Current status | Evidence |
|---|---:|---|
| Sidebar logger recursion | Fixed | `assets/js/sidebar.js:16-20` now calls `console.log/error/info/warn`. |
| Core post-scoped generation auth | Partially fixed | Chat, generate-plan, execute-article, reformat-block, and regenerate-block now check post permissions, but other post-scoped routes still do not. |
| Clear-context active session clearing | Backend fixed, auth still open | Context service clears all active sessions for the post when no `sessionId` is provided at `includes/class-context-service.php:324-345`; REST handler lacks target-post permission. |
| Legacy chat-history helper writes | Mostly fixed | `update_post_chat_history()` is now a no-op at `includes/class-gutenberg-sidebar.php:1313-1317`; migration still does not delete legacy meta or run on read. |
| WP AI wrapper cost metadata | Improved | Wrapper uses `record_usage_full()` at `includes/class-wp-ai-client-wrapper.php:197-207` and `254-264`. |
| Cost table missing-table self-heal | Fixed | `maybe_upgrade_table()` checks `SHOW TABLES LIKE` and recreates the table at `includes/class-cost-tracker.php:71-80`. |
| Provider metadata outside chat | Still open | Provider metadata is still not a uniform response contract across non-chat endpoints. |
| Model registry/default unification | Still open | Defaults remain spread across activation, settings PHP/JS, providers, image manager, and WP AI wrapper. |
## Critical Findings
### P0: Several Post-Scoped REST Handlers Still Lack Target `edit_post` Authorization
The previous pass improved a subset of handlers, but the larger REST surface remains inconsistent. These handlers still accept or derive a post id and then read, write, or analyze post-scoped data without a target-post permission check, or they check too late:
- `handle_clear_context()` deletes post meta and clears sessions for `postId`, but only validates that the id is positive. It does not call `check_post_permission()` before clearing context at `includes/class-gutenberg-sidebar.php:1240-1254`.
- `handle_revise_plan()` reads post config, detected language, post memory, and writes `_wpaw_plan`, `_wpaw_detected_language`, and `_wpaw_memory` without an upfront post permission check at `includes/class-gutenberg-sidebar.php:2014-2134`.
- `handle_block_refine()` reads `_wpaw_plan` and tracks cost against `postId` without checking post permissions at `includes/class-gutenberg-sidebar.php:4185-4313`.
- `handle_refine_from_chat()` passes `postId` into a streaming refinement path without checking post permissions at `includes/class-gutenberg-sidebar.php:4830-4849`.
- `handle_seo_audit()` reads post content and post config without checking post permissions at `includes/class-gutenberg-sidebar.php:5714-5735`.
- `handle_generate_meta()` reads post content before checking permissions; the permission check happens only after content/title are copied at `includes/class-gutenberg-sidebar.php:6032-6057`.
- `handle_suggest_keywords()` reads detected language and post config for `postId` without checking post permissions at `includes/class-gutenberg-sidebar.php:6155-6179`.
- `handle_suggest_improvements()` reads and parses post content and post config without checking permissions at `includes/class-gutenberg-sidebar.php:6393-6451`.
Impact:
- Authors/editors with generic `edit_posts` can still potentially read or mutate plugin data for posts they cannot edit.
- The clear-context endpoint can delete another post's agent memory and conversation sessions.
- SEO/GEO/suggestion routes can leak content, focus keywords, detected language, post config, memory, or outlines.
- Cost tracking can still be attributed to unauthorized posts from some flows.
Recommended fix:
- Add a single helper such as `require_post_permission_from_params( $params, 'postId' )`.
- Call it before any `get_post()`, `get_post_meta()`, `get_post_config()`, `get_post_memory_context()`, `update_post_meta()`, cost attribution, or streaming start.
- For read-only analysis endpoints, still require `current_user_can( 'edit_post', $post_id )` because these routes expose draft/private/editorial data.
- Add negative permission tests for at least clear-context, revise-plan, SEO audit, generate-meta, block-refine, and suggest-improvements.
## High Priority Findings
### P1: Legacy Chat History Is Deprecated But Still Not Fully Migrated or Retired
Legacy write paths are improved, but the old store still exists:
- `/chat-history/(?P<post_id>\d+)` remains registered and returns `_wpaw_chat_history`, although it now marks the response as deprecated at `includes/class-gutenberg-sidebar.php:1264-1299`.
- `migrate_legacy_chat_history()` still keeps `_wpaw_chat_history` after migration because deletion is commented out at `includes/class-context-service.php:299-300`.
- The context service header says legacy chat history is migrated on read, but `get_context()` still returns an empty context when no session exists and does not trigger migration at `includes/class-context-service.php:62-87`.
Impact:
- The plugin still has two possible history stores.
- Legacy meta can be re-imported or returned after a session is cleared or migrated.
- Chat/context work remains more fragile than it needs to be.
Recommended fix:
- Decide the final legacy policy:
- Delete `_wpaw_chat_history` after successful migration, or
- Write a durable `_wpaw_chat_history_migrated` marker and never re-import it.
- Add migrate-on-read for post-linked contexts.
- Remove or hard-disable `/chat-history` once the sidebar fully uses conversations.
### P1: Provider Metadata Is Still Not a Uniform Response Contract
Chat responses include provider metadata, but most non-chat generated responses still do not expose it consistently. The code tracks provider metadata in some cost hooks, but the API response contract remains inconsistent for plan, revise, execute, block refinement, keyword, image, SEO/GEO, and suggestion flows.
Impact:
- Users cannot reliably tell which provider/model actually served a non-chat request.
- Fallback debugging remains harder than necessary.
- Cost records and UI messages can diverge.
Recommended fix:
- Standardize a generated response envelope:
- `provider`
- `selected_provider`
- `fallback_used`
- `warnings`
- `model`
- `cost`
- Apply it to all generated text/image endpoints and streaming completion events.
## Medium Priority Findings
### P2: Legacy `record_usage()` Still Defaults Provider to OpenRouter
Current internal wrapper calls now use `record_usage_full()`, so the third-pass attribution bug is mostly closed. The older `record_usage()` compatibility method still records provider as `openrouter` at `includes/class-cost-tracker.php:175-186`.
Impact:
- Any future or external caller using `record_usage()` can still create misleading provider rows.
Recommended fix:
- Mark `record_usage()` as deprecated in the docblock.
- Either require a provider argument or route it through `record_usage_full()` with an explicit `provider='unknown'`.
- Prefer converting all internal calls to `record_usage_full()`.
### P2: Model Defaults Remain Fragmented
Model defaults are still spread across:
- Activation defaults in `wp-agentic-writer.php`.
- Sidebar localized defaults in `includes/class-gutenberg-sidebar.php`.
- Settings PHP defaults in `includes/class-settings.php` and `includes/class-settings-v2.php`.
- JS presets in `assets/js/settings-v2.js`.
- Provider defaults in OpenRouter, local backend, Codex, and WP AI wrapper classes.
- Image defaults in `includes/class-image-manager.php`.
Impact:
- A model change can fix one runtime path while leaving another path stale.
- Settings UI, provider runtime, and cost analytics can disagree about the active model.
Recommended fix:
- Introduce a PHP model registry/default resolver.
- Localize resolved defaults to JS instead of duplicating them.
- Treat legacy `execution_model` as a migration alias into canonical `writing_model`.
### P2: Sidebar WordPress Compatibility Still Needs Browser Verification
The sidebar imports `PluginSidebar` from `wp.editPost` at `assets/js/sidebar.js:8-10`. This may be fine for the target WordPress version, but the previous fallback was removed.
Recommended fix:
- Browser-test the sidebar on the minimum supported WordPress version.
- Restore the fallback if `wp.editPost.PluginSidebar` is not guaranteed.
## Definition of Done Gates for This Pass
Before considering this retrace complete:
- Every REST handler that accepts or derives a post id checks `edit_post` before post reads, post writes, streaming, or cost attribution.
- Clear-context cannot clear sessions or meta for a post the current user cannot edit.
- Legacy `_wpaw_chat_history` migration either deletes the legacy meta or writes a durable migration marker.
- `get_context()` behavior matches its documented migrate-on-read rule.
- Provider metadata is returned consistently for non-chat generated responses.
- PHP and JS syntax checks remain green.
## Current Decision
The third-pass implementation is meaningfully better, but the plugin is still not clean enough to shift fully into new chat/context work. Finish the remaining post-scoped authorization sweep first, then close the legacy history migration gap. After that, chat/context implementation will be much less likely to keep reopening old regressions.

194
docs/architecture/PLUGIN_AUDIT_RETRACE_NINTH_PASS_2026-05-26.md Normal file
View File

@@ -0,0 +1,194 @@
# WP Agentic Writer Ninth Retrace Audit
Status: COMPLETE / RETRACED
Completion marker: 2026-05-26
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_TENTH_PASS_2026-05-26.md`
Audit date: 2026-05-26
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_EIGHTH_PASS_2026-05-25.md`
Scope: ninth pass after eighth-retrace implementation, covering chat/context continuity, failed-attempt tracking, provider transparency, model registry adoption, UI/UX, and release readiness.
## Executive Summary
The eighth-pass implementation closed several important items:
- The `/chat-history` compatibility path no longer depends on `_wpaw_active_session_id`; it now uses `get_sessions_for_post()` and the migrated session id, so the empty-history-after-migration bug appears fixed.
- A `track_ai_cost()` helper now exists and failed AI branches were added for several previously invisible error paths.
- A new `includes/class-model-registry.php` file exists and activation defaults now use it.
- Successful backend cost hooks and backend provider metadata coverage remain much better than earlier passes.
- PHP and JavaScript syntax checks pass.
One serious runtime regression remains: several new failed-attempt paths call `$provider_result->get_default_model()`, but `WPAW_Provider_Selection_Result` does not define that method. That will turn provider failures into fatal PHP errors instead of graceful error handling and cost tracking.
The remaining non-fatal gaps are mostly contract adoption: the sidebar still hydrates chat through the deprecated `/chat-history` route, provider metadata is still not rendered in the editor, and the model registry exists but is not yet the actual single source of truth across settings, JS presets, providers, and image helpers.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of eighth-pass findings against current code.
- Static sweep of chat-history migration, failed-attempt tracking, provider metadata UI usage, model registry adoption, and model default duplication.
- No live WordPress editor/browser workflow was run in this pass.
## Eighth-Pass Status Trace
| Eighth-pass item | Current status | Evidence |
|---|---:|---|
| `/chat-history` empty after migration | Fixed | `get_post_chat_history()` now uses `get_sessions_for_post()` and the returned `$migrated_session_id` at `includes/class-gutenberg-sidebar.php:1397-1434`. |
| Sidebar dependency on `/chat-history` | Still open as cleanup/API debt | `assets/js/sidebar.js:644-668` still fetches `/chat-history/${postId}`. |
| Failed-attempt cost tracking | Regressed in runtime error paths | Failed branches call nonexistent `$provider_result->get_default_model()` at `includes/class-gutenberg-sidebar.php:3798`, `4165`, `7006`, and `7104`. |
| Cost helper | Added | `track_ai_cost()` exists at `includes/class-gutenberg-sidebar.php:966-1005`. |
| Provider metadata backend coverage | Improved | Backend responses include `provider_metadata` in more places, and chat now also adds a `provider_metadata` envelope at `includes/class-gutenberg-sidebar.php:1114`. |
| Provider metadata UI rendering | Still open | `assets/js/sidebar.js` has no references to `provider_metadata`, `fallback_used`, `selected_provider`, or `warnings`; only web-search provider checks exist at `assets/js/sidebar.js:5983-6000`. |
| Model registry | Created, partial adoption | `includes/class-model-registry.php` exists and is required by `wp-agentic-writer.php:50-51`; activation uses it at `wp-agentic-writer.php:140-145`. |
| Model registry as single source of truth | Still open | Settings, JS presets, provider defaults, and image helper fallbacks still hard-code model ids. |
| WordPress editor browser pass | Still open | Syntax checks passed, but no live editor workflow was verified. |
## Remaining Findings
### P0: Failed AI Paths Can Fatal On A Nonexistent Provider-Result Method
The new failed-attempt tracking calls this pattern:
- `includes/class-gutenberg-sidebar.php:3796-3805` for regeneration.
- `includes/class-gutenberg-sidebar.php:4163-4172` for clarity API failure.
- `includes/class-gutenberg-sidebar.php:7004-7012` for multi-pass refinement.
- `includes/class-gutenberg-sidebar.php:7102-7110` for article refinement.
Each uses:
```php
$provider_result->get_default_model() ?? 'unknown'
```
But `WPAW_Provider_Selection_Result` only defines public properties and a constructor at `includes/class-provider-manager.php:20-33`; it has no `get_default_model()` method. The only `get_default_model()` found is the static registry method `WPAW_Model_Registry::get_default_model( $task )` at `includes/class-model-registry.php:131-134`.
Impact:
- A provider error in these routes can produce a fatal PHP error.
- The intended graceful fallback/error response will not run.
- The intended failed-attempt cost tracking may not record anything.
- This is especially risky because it triggers exactly when providers are unavailable, misconfigured, or failing.
Recommended fix:
- Replace the invalid method call with task-specific registry calls:
- Regeneration: `WPAW_Model_Registry::get_default_model( 'writing' )`
- Clarity: `WPAW_Model_Registry::get_default_model( 'clarity' )`
- Multi-pass/article refinement: `WPAW_Model_Registry::get_default_model( 'refinement' )`
- Or pass the model explicitly into the failed-attempt helper from the request context/provider.
- Add a focused regression check that simulates `is_wp_error( $response )` for each route and asserts no fatal occurs.
### P1: Provider Transparency Still Does Not Reach The Editor UI
Backend metadata improved. `build_provider_metadata()` returns provider, selected provider, fallback status, warnings, and model at `includes/class-gutenberg-sidebar.php:956-963`, and chat now also adds `provider_metadata` at `includes/class-gutenberg-sidebar.php:1114`.
The editor still does not render that data:
- `assets/js/sidebar.js` has no references to `provider_metadata`, `fallback_used`, `selected_provider`, or `warnings`.
- The only provider-related sidebar references are web-search availability checks at `assets/js/sidebar.js:5983-6000`.
Impact:
- Users still cannot see actual provider/model/fallback behavior in the editor.
- The Definition of Done requires the UI to show actual provider used.
- Backend metadata can help logs/API consumers, but it does not yet close the UX transparency gap.
Recommended fix:
- Add sidebar state for provider metadata from chat, plan, write, refine, meta, and utility AI responses.
- Render a compact provider/model/fallback line near the cost/status UI.
- Show warnings when fallback occurs.
- Verify with a browser/editor pass.
### P1: Model Registry Exists But Is Not Yet The Single Source Of Truth
The new registry is a good foundation:
- `includes/class-model-registry.php:24-219` defines task defaults, fallbacks, labels, frontend data, and activation defaults.
- The plugin requires it at `wp-agentic-writer.php:50-51`.
- Activation uses `WPAW_Model_Registry::get_activation_defaults()` at `wp-agentic-writer.php:140-145`.
But several runtime surfaces still hard-code defaults independently:
- Settings V2 localization still falls back to old `google/gemini-2.0-flash-exp:free` values at `includes/class-settings-v2.php:100-111`.
- Settings V2 sanitization still hard-codes defaults at `includes/class-settings-v2.php:988-994`.
- JavaScript presets are still hard-coded in `assets/js/settings-v2.js:32-58`.
- OpenRouter provider properties are still hard-coded at `includes/class-openrouter-provider.php:29-75`; comments say registry-sourced, but the constructor uses the hard-coded property values when settings are absent at `includes/class-openrouter-provider.php:437-448`.
- Image manager still hard-codes `anthropic/claude-3.5-sonnet` and `openai/gpt-4o` fallbacks at `includes/class-image-manager.php:183-249`.
Impact:
- The registry can drift from runtime behavior.
- Settings UI can show or save defaults that do not match activation/provider defaults.
- The screenshot's "model registry complete" claim is directionally true for creation, but not yet true for full adoption.
Recommended fix:
- Replace settings fallback literals with `WPAW_Model_Registry::get_default_model()`.
- Localize `WPAW_Model_Registry::get_frontend_data()` to settings JS and derive presets from PHP data or explicitly document presets as curated overrides.
- Initialize provider defaults from the registry instead of matching literals manually.
- Replace image manager model fallbacks with registry calls.
- Add a consistency test that fails if hard-coded task default strings appear outside the registry or approved presets.
### P2: Sidebar Still Uses Deprecated Chat-History Route
The data-loss-style bug is fixed because `get_post_chat_history()` now uses sessions and migrated session ids. The remaining issue is contract cleanliness:
- `assets/js/sidebar.js:644-668` still calls `/chat-history/${postId}`.
- The endpoint docblock still says it "does not use the conversations table" at `includes/class-gutenberg-sidebar.php:1337-1339`, but the implementation now does use conversations.
Impact:
- The frontend still depends on a deprecated compatibility endpoint.
- Documentation and behavior disagree.
- Future refactors may remove or alter the route, breaking chat hydration again.
Recommended fix:
- Move sidebar hydration to the canonical conversation/session context endpoint.
- If `/chat-history` remains, update the docblock and response to include the canonical `session_id` and an explicit `source`.
### P2: Cost Tracking Is Better But Still Allows Raw Hook Drift
`track_ai_cost()` exists, but many raw `do_action( 'wp_aw_after_api_request', ... )` calls still remain in `includes/class-gutenberg-sidebar.php`.
Impact:
- New or edited routes can bypass the helper and reintroduce missing provider/session/status data.
- The failed-attempt fatal bug demonstrates why centralizing error/success tracking matters.
Recommended fix:
- Convert remaining raw hook calls in `class-gutenberg-sidebar.php` to `track_ai_cost()`.
- Add a static check that disallows direct `wp_aw_after_api_request` calls outside `track_ai_cost()` and the cost tracker registration.
### P2: Editor UI/UX Browser Verification Still Remains
The screenshot correctly identifies browser verification as remaining. Static checks passed, but no live editor workflow was run.
Recommended browser checklist:
- Sidebar opens and persists in the block editor.
- Chat session continues after page reload.
- Provider/fallback warnings render when metadata exists.
- Cost display updates after chat, plan, refine, and meta actions.
- Unauthorized post access fails cleanly.
- Model settings changes reflect in generated requests.
## Recommended Next Work
1. Fix the nonexistent `$provider_result->get_default_model()` calls immediately.
2. Render provider/model/fallback/warnings in the sidebar.
3. Finish model registry adoption across settings, JS, provider defaults, and image manager fallbacks.
4. Move sidebar chat hydration off `/chat-history`, or update the compatibility contract and docblock.
5. Convert raw cost hooks to `track_ai_cost()` and add a static guard.
6. Run the live WordPress editor browser workflow pass.
## Current Verdict
The eighth-pass implementation is partially proper and closes the prior chat-history migration bug. It also added useful infrastructure with `track_ai_cost()` and `WPAW_Model_Registry`.
It is not release-clean yet. The top blocker is the failed-attempt runtime fatal caused by calling `get_default_model()` on the wrong object. After that, the remaining work is mostly contract completion: provider metadata in UI, full registry adoption, and browser verification.

238
docs/architecture/PLUGIN_AUDIT_RETRACE_SECOND_PASS_2026-05-24.md Normal file
View File

@@ -0,0 +1,238 @@
# WP Agentic Writer Second Retrace Audit
Status: COMPLETE / SUPERSEDED
Completion marker date: 2026-05-24
Next retrace report: `docs/architecture/PLUGIN_AUDIT_RETRACE_THIRD_PASS_2026-05-24.md`
Audit date: 2026-05-24
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_2026-05-24.md`
Scope: second pass after retrace implementation, covering UI/UX, editor runtime, system boundaries, conversation context/history, cost tracker, provider/model routing, migrations, data lifecycle, and release readiness.
## Executive Summary
Several of the previous retrace findings were implemented correctly. The provider selection result is now unwrapped in the image, keyword, and WP AI legacy provider paths. Streaming chat state variables are initialized again. The cost table base schema and hook argument count were expanded. Uninstall cleanup is also much more complete, and a changelog now exists.
However, the plugin is not ready to move only into chat/context feature work yet. Two release blockers remain:
1. `assets/js/sidebar.js` currently does not parse. The attempted debug-logger conversion damaged string literals and logger calls, so the Gutenberg sidebar can fail before any UI renders.
2. `includes/class-wp-ai-client-wrapper.php` still calls `WP_Agentic_Writer_Cost_Tracker::record_usage()`, but that method does not exist. Successful WP AI Client text generation or legacy wrapper tracking can fatal.
After those are fixed, the next biggest risks are still the conversation migration/version gate, legacy context source-of-truth behavior, and post-scoped authorization gaps outside the conversation routes.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar.js`: failed at line 17.
- Static trace of prior retrace recommendations against the current code.
- No live WordPress browser workflow was run in this pass because the sidebar JavaScript parse failure blocks meaningful UI verification.
## Retrace Implementation Status
| Area | Current status | Evidence |
|---|---:|---|
| Provider selection result callers | Fixed for previously named helper paths | Image manager, keyword suggester, and WP AI legacy wrapper now use `$provider_result->provider`. |
| Streaming chat variables | Fixed | `stream_chat_request()` initializes accumulated content, chunk count, total cost, and last user message before streaming. |
| Cost hook expanded args | Fixed | `includes/class-cost-tracker.php:48-53` registers `add_request` with 9 accepted args. |
| Cost base schema | Improved | `wp-agentic-writer.php:198-216` now creates `session_id`, `provider`, and `status` columns and indexes. |
| Uninstall/data cleanup | Improved | Main uninstall removes settings, custom models, tables, transients, user meta, post meta, scheduled events, and temp images. |
| Changelog | Added but currently inaccurate | `CHANGELOG.md` says sidebar logging was fixed, but `assets/js/sidebar.js` now fails syntax validation. |
| Conversation migration versioning | Still open | Conversation migration still checks `wpaw_db_version` instead of `wpaw_conversations_db_version`. |
| Context/history source of truth | Still partial | Legacy migration is manual, `get_context()` does not migrate on read, and clear-context route only deletes post meta. |
| Post-scoped REST authorization | Still partial | Several post-id routes rely on generic `edit_posts` route permission and do not check `edit_post` for the target post. |
| Model registry/default unification | Still open | Defaults and model lists remain spread across activation, settings PHP, settings JS, providers, and wrapper classes. |
## Critical Findings
### P0: Gutenberg Sidebar JavaScript Does Not Parse
The sidebar bundle currently fails before runtime:
- `node -c assets/js/sidebar.js` fails with `SyntaxError: Invalid or unexpected token`.
- The first parse error is at `assets/js/sidebar.js:17`, where the logger calls `wpawLog.log(', ...args);`.
- `assets/js/sidebar.js:18` has the same pattern for errors.
- Many later converted calls lost opening quotes or brackets, for example `wpawLog.error( Failed to load session for post:', error);`, `wpawLog.log( Found legacy chat history, triggering migration...');`, and `wpawLog.warn([WPAW] No questions returned from clarity check!);`.
Impact:
- The editor sidebar can fail to load entirely.
- Chat, context migration prompts, cost UI refresh, plan generation, article generation, and image UI are all unreachable if the script is enqueued.
- This also invalidates the changelog claim that the sidebar debug logging conversion is complete.
Recommended fix:
- Restore the logger definition to call `console.log`, `console.error`, `console.info`, and `console.warn`.
- Repair every malformed `wpawLog.*` call in `assets/js/sidebar.js`.
- Prefer a mechanical but syntax-aware conversion from `console.*` to `wpawLog.*`, not a plain text replacement.
- Gate completion on `node -c assets/js/sidebar.js` plus a browser load of the Gutenberg sidebar.
### P0: WP AI Wrapper Still Calls Missing Cost Tracker Method
`includes/class-wp-ai-client-wrapper.php` still calls a method that does not exist:
- Core AI text path calls `WP_Agentic_Writer_Cost_Tracker::get_instance()->record_usage()` at `includes/class-wp-ai-client-wrapper.php:197-202`.
- Legacy text path calls the same missing method at `includes/class-wp-ai-client-wrapper.php:249-254`.
- Repository search found no `record_usage()` implementation in `WP_Agentic_Writer_Cost_Tracker`.
Impact:
- If WP AI Client text generation succeeds and cost tracking class is loaded, the request can fatal after the model returns.
- If the legacy wrapper succeeds and tries to track cost, it can fatal.
- This is a cross-path cost-tracker regression because most of the plugin uses `do_action( 'wp_aw_after_api_request', ... )`, but this wrapper uses a different contract.
Recommended fix:
- Either add a `record_usage()` compatibility method to `WP_Agentic_Writer_Cost_Tracker`, or convert both wrapper call sites to `do_action( 'wp_aw_after_api_request', ... )`.
- The compatibility method is safer if external or older internal code might call it.
- Include model, action/task type, input/output token estimates, cost, provider, session id, and status where available.
## High Priority Findings
### P1: Conversation Migration Still Uses the Main DB Version Gate
The conversation migration stores a dedicated option but does not consistently read it:
- `wpaw_create_conversations_table()` stores `wpaw_conversations_db_version` at `includes/class-conversation-migration.php:43-47`.
- `wpaw_run_migrations()` still reads `wpaw_db_version` at `includes/class-conversation-migration.php:67-72`.
- Main table creation only runs when `wpaw_db_version < 1.1.0` in `wp-agentic-writer.php:230-249`.
Impact:
- Sites where the main plugin DB version is already current can skip conversation table repair or creation.
- A failed partial migration can leave the plugin thinking the main schema is current while conversation storage is missing or stale.
- Chat/history work remains risky until table creation is idempotent per table/version, not only per global plugin version.
Recommended fix:
- Make conversation table creation check `wpaw_conversations_db_version`.
- In `wp_agentic_writer_maybe_create_tables()`, either always call idempotent table creators or use per-table schema versions.
- Update `wpaw_drop_conversations_table()` to delete `wpaw_conversations_db_version`, not `wpaw_db_version`.
### P1: Clear Context and Legacy Migration Still Do Not Respect the New Source of Truth
The code still has split context behavior:
- `WP_Agentic_Writer_Context_Service::get_context()` returns an empty context when no session exists and does not try legacy migration on read.
- `migrate_legacy_chat_history()` keeps `_wpaw_chat_history` after migration at `includes/class-context-service.php:299-300`.
- `handle_clear_context()` only deletes `_wpaw_memory` and `_wpaw_chat_history` at `includes/class-gutenberg-sidebar.php:1216-1236`; it does not clear the active conversation session.
- The context service has a better `clear_context( $session_id, $post_id )` method, but the REST handler does not use it.
Impact:
- Users can clear visible legacy meta while the active session still keeps messages.
- Legacy meta can be re-migrated later, causing duplicate or surprising history.
- The system still has two competing history stores, which is the core reason A/B regressions keep recurring around chat.
Recommended fix:
- Pass `sessionId` through the clear-context endpoint and call `WP_Agentic_Writer_Context_Service::clear_context()`.
- Decide migration policy: either delete legacy `_wpaw_chat_history` after successful migration or mark it migrated with a durable meta flag.
- Add migrate-on-read for post-linked sessions so old posts behave consistently.
### P1: Post-Scoped REST Authorization Is Still Incomplete Outside Conversation Routes
The conversation routes received targeted post authorization, but older post-id endpoints still depend mostly on generic `edit_posts` permission:
- Route registration uses `check_permissions`, which returns `current_user_can( 'edit_posts' )` at `includes/class-gutenberg-sidebar.php:931`.
- `handle_get_post_config()` and `handle_update_post_config()` do not check `edit_post` for the target post at `includes/class-gutenberg-sidebar.php:1728-1760`.
- `handle_get_cost_tracking()` does not check target post access at `includes/class-gutenberg-sidebar.php:3629-3635`.
- `handle_save_section_blocks()` and `handle_get_section_blocks()` do not check target post access at `includes/class-gutenberg-sidebar.php:4782-4842`.
- Image recommendation/generation/commit handlers do not check target post access at `includes/class-gutenberg-sidebar.php:6463-6525`.
Impact:
- An authenticated author/editor with broad `edit_posts` can potentially read or mutate plugin data for posts they cannot edit.
- Cost history, post config, image recommendations, image variants, and section mappings can leak or be modified across post boundaries.
Recommended fix:
- Every REST handler accepting `post_id` or `postId` should validate `current_user_can( 'edit_post', $post_id )` before read or write.
- Centralize this in helper methods so future endpoints inherit the same rule.
- Include negative permission tests for another user's private/draft post.
## Medium Priority Findings
### P2: Cost Tracker Migration Still Assumes the Table Exists
`WP_Agentic_Writer_Cost_Tracker::maybe_upgrade_table()` runs `DESCRIBE {$table_name}` and immediately uses `in_array()` on the result. If the table is missing, `$wpdb->get_col()` can return an unexpected value and the method does not create the table.
Impact:
- Sites with a missing cost table but a current `wpaw_db_version` may not self-heal.
- The first cost tracker access can produce warnings or fail to record usage.
Recommended fix:
- Detect missing table with `SHOW TABLES LIKE` before `DESCRIBE`.
- If missing, call `wp_agentic_writer_create_cost_table()` or move schema creation into the cost tracker class.
- Make the table creator idempotent and callable regardless of global plugin DB version.
### P2: Provider Metadata Is Still Not a Uniform Response Contract
Chat responses now include provider metadata, but non-chat endpoints still often return only content or variants. As provider fallback expands, users and support logs need a consistent answer to "which provider/model actually served this?".
Recommended fix:
- Standardize response metadata for generated text, image analysis, image variants, keyword suggestions, and plan/edit endpoints.
- Include `provider`, `fallback_used`, `warnings`, `model`, and `cost` where available.
- Mirror the same metadata into cost records.
### P2: Model Defaults and Registry Remain Fragmented
Model defaults still appear in multiple layers:
- Activation defaults in `wp-agentic-writer.php`.
- Settings presets in `assets/js/settings-v2.js`.
- Settings PHP model transforms and saved options.
- Provider defaults in OpenRouter/local/WP AI wrapper classes.
- Image model fallback logic in the image manager.
Impact:
- A model replacement can fix one UI path while leaving old defaults in runtime paths.
- Cost estimation and provider selection may disagree with the model shown in settings.
Recommended fix:
- Create a single model registry/default resolver in PHP.
- Localize the resolved defaults to JS instead of duplicating presets.
- Give each task type one canonical key: chat, clarity, planning, writing, refinement, image.
### P2: Deactivation Still Leaves the Conversation Cleanup Event Scheduled
Uninstall now clears `wpaw_cleanup_old_sessions`, but deactivation only clears `wpaw_cleanup_temp_images` in `wp-agentic-writer.php:260-266`.
Impact:
- A deactivated plugin can leave scheduled cleanup hooks behind until uninstall.
Recommended fix:
- Add `wp_clear_scheduled_hook( 'wpaw_cleanup_old_sessions' )` to deactivation.
- Prefer scheduling cron events on activation or migration, not at include time.
## Definition of Done Gates for This Pass
Before considering the retrace implementation complete, require:
- `node -c assets/js/sidebar.js` passes.
- `node -c assets/js/settings-v2.js` passes.
- Full PHP syntax check passes.
- No references to undefined methods such as `record_usage()` remain, unless the method is intentionally added.
- Conversation table creation is controlled by `wpaw_conversations_db_version` or always-idempotent table checks.
- Clear-context clears the active session and legacy meta in one path.
- Every post-scoped endpoint checks `edit_post` for its target post.
- Changelog entries match verified behavior.
## Recommended Next Work Order
1. Fix `assets/js/sidebar.js` syntax and logger conversion, then browser-test the sidebar.
2. Fix the missing `record_usage()` contract or convert the WP AI wrapper to the existing cost hook.
3. Fix conversation table versioning and table self-healing.
4. Align clear-context and legacy migration around the conversations table as the source of truth.
5. Add post-scoped authorization checks for config, cost, section-block, and image routes.
6. Then continue into deeper chat/context implementation.
## Current Decision
Do not move fully into new chat/context feature work yet. The sidebar parse failure and missing cost tracker method should be fixed first because they can block or fatal the existing editor workflow. Once those P0s are closed, chat/context can become the main focus with fewer regressions.

208
docs/architecture/PLUGIN_AUDIT_RETRACE_SEVENTH_PASS_2026-05-25.md Normal file
View File

@@ -0,0 +1,208 @@
# WP Agentic Writer Seventh Retrace Audit
Status: COMPLETE / RETRACED
Completion marker: 2026-05-25
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_EIGHTH_PASS_2026-05-25.md`
Audit date: 2026-05-25
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_SIXTH_PASS_2026-05-25.md`
Scope: seventh pass after sixth-retrace implementation, covering provider transparency, conversation context/history, cost tracking, model defaults, UI/UX, and release readiness.
## Executive Summary
The sixth-pass implementation fixed two important backend issues:
- `WP_Agentic_Writer_Context_Service::get_context()` now carries an `$effective_session_id` and returns it, so migrate-on-read no longer returns messages from one session while reporting the original stale session id.
- Deprecated `WP_Agentic_Writer_Cost_Tracker::record_usage()` now records provider as `unknown` instead of hard-coding `openrouter`.
I did not find a new P0 authorization blocker. The remaining problems are still meaningful because they affect user trust, cost accuracy, and conversation continuity:
- The sidebar still actively calls deprecated `/chat-history`, which reads legacy `_wpaw_chat_history` instead of the authoritative conversation table.
- Provider transparency was added only partially: some responses now include nested `provider_metadata`, but other AI endpoints still omit it, chat uses a different top-level shape, and the sidebar does not render provider/fallback metadata.
- Cost tracking still defaults missing provider arguments to `openrouter`, and many AI cost hooks still omit provider/session/status arguments.
- Model defaults remain fragmented across PHP activation, settings, providers, JavaScript presets, wrappers, and image helpers.
- Browser-level WordPress editor verification is still not done.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of sixth-pass findings against current code.
- Static sweep of provider metadata, cost hooks, context migration, sidebar history loading, and model defaults.
- No live WordPress editor/browser workflow was run in this pass.
## Sixth-Pass Status Trace
| Sixth-pass item | Current status | Evidence |
|---|---:|---|
| Migrated context returns effective session id | Fixed | `$effective_session_id` is initialized, updated after migration, and returned as `session_id` in `includes/class-context-service.php:62-92`. |
| Deprecated `record_usage()` provider attribution | Fixed narrowly | The deprecated wrapper now passes `unknown` at `includes/class-cost-tracker.php:176-186`. |
| Provider metadata helper | Partially fixed | `build_provider_metadata()` exists at `includes/class-gutenberg-sidebar.php:956-963`, and several responses now include `provider_metadata`. |
| Provider transparency response contract | Still partial | Chat uses top-level keys, several non-chat routes use nested `provider_metadata`, and some AI routes still omit metadata entirely. |
| Legacy `/chat-history` endpoint | Still open, now confirmed active in UI | Backend still reads `_wpaw_chat_history`, and `assets/js/sidebar.js` still calls `/chat-history/{postId}` on load. |
| Cost tracker provider/session/status integrity | Still open | The cost hook accepts 9 args, but many call sites still pass only 7 args, causing provider fallback to the hook default. |
| Model registry/default unification | Still open | No model registry was found; defaults remain duplicated across runtime surfaces. |
| WordPress editor browser pass | Still open | Syntax checks passed, but no editor workflow was verified. |
## Remaining Findings
### P1: Sidebar Still Loads Conversation State From Deprecated Post Meta
The backend still registers and serves the legacy route:
- `/chat-history/(?P<post_id>\d+)` remains registered in `includes/class-gutenberg-sidebar.php:346-354`.
- `handle_get_chat_history()` still returns `messages` from `get_post_chat_history()` in `includes/class-gutenberg-sidebar.php:1300-1326`.
- `get_post_chat_history()` still reads `_wpaw_chat_history` directly in `includes/class-gutenberg-sidebar.php:1354-1364`.
More importantly, this is still an active frontend dependency:
- `assets/js/sidebar.js:644-668` calls `${wpAgenticWriter.apiUrl}/chat-history/${postId}` and seeds `messages` from that response.
Impact:
- The Definition of Done says conversation messages are authoritative in `wpaw_conversations`, but the sidebar can still hydrate UI state from legacy post meta.
- Migrated or newly created sessions can look empty while old post-meta history appears, or old history can override the expected session model on first render.
- This keeps two mental models alive: "chat belongs to a post meta array" and "chat belongs to a conversation session."
Recommended fix:
- Replace the sidebar history load with a conversation-session/context endpoint.
- If compatibility is still needed, make `/chat-history` return session-backed data only, not raw `_wpaw_chat_history`.
- Add a regression check that `assets/js/sidebar.js` has no `/chat-history` fetch after migration.
### P1: Cost Provider Attribution Is Still Wrong For Many AI Actions
The deprecated `record_usage()` wrapper was fixed, but the main action hook still defaults missing provider data to OpenRouter:
- `add_action( 'wp_aw_after_api_request', ..., 10, 9 )` expects provider/session/status arguments in `includes/class-cost-tracker.php:48-50`.
- `add_request()` still defaults `$provider = 'openrouter'` in `includes/class-cost-tracker.php:124`.
Many AI routes still call `do_action( 'wp_aw_after_api_request', ... )` with only the first seven arguments, so the cost table will store `openrouter` even when the actual provider was local backend, Codex, or another fallback:
- Clarity check at `includes/class-gutenberg-sidebar.php:4096-4106`.
- Meta description at `includes/class-gutenberg-sidebar.php:6217-6228`.
- Summarize context at `includes/class-gutenberg-sidebar.php:6393-6402`.
- Multi-pass refinement at `includes/class-gutenberg-sidebar.php:6868-6877`.
- Article refinement at `includes/class-gutenberg-sidebar.php:6954-6963`.
- Execution total at `includes/class-gutenberg-sidebar.php:3244-3253`.
- Regeneration at `includes/class-gutenberg-sidebar.php:3716-3725`.
Impact:
- The visible response can say one provider while `wpaw_cost_tracking.provider` records another.
- Cost review, provider debugging, local/cloud usage reporting, and fallback analysis become unreliable.
- This directly undermines the cost counter/tracker area the audit chain is trying to stabilize.
Recommended fix:
- Change the hook default provider from `openrouter` to `unknown`.
- Add provider/session/status to every AI `wp_aw_after_api_request` call.
- Add a small helper like `track_ai_cost( $post_id, $response, $action, $provider_result, $session_id = '', $status = 'success' )` so new routes cannot omit metadata accidentally.
- Add one test or static check that no AI cost hook call has only seven payload arguments.
### P1: Provider Transparency Is Present But Inconsistent And Not Surfaced In UI
A shared helper now exists:
- `build_provider_metadata()` returns `provider`, `selected_provider`, `fallback_used`, `warnings`, and `model` at `includes/class-gutenberg-sidebar.php:956-963`.
Several responses now include nested `provider_metadata`, for example:
- Plan generation at `includes/class-gutenberg-sidebar.php:2024-2033`.
- Plan revision at `includes/class-gutenberg-sidebar.php:2189-2197`.
- Block refinement at `includes/class-gutenberg-sidebar.php:4384-4393`.
- Summarize context at `includes/class-gutenberg-sidebar.php:6404-6414`.
But the contract is not yet consistent:
- Chat responses still write top-level `provider`, `selected_provider`, `fallback_used`, and `warnings` at `includes/class-gutenberg-sidebar.php:1067-1071`.
- Some non-chat AI responses still omit provider metadata entirely, including execution response at `includes/class-gutenberg-sidebar.php:3255-3260`, regenerate block response at `includes/class-gutenberg-sidebar.php:3727-3732`, and article refinement response at `includes/class-gutenberg-sidebar.php:6965-6970`.
- `assets/js/sidebar.js` has no handling for `provider_metadata`, `fallback_used`, or `warnings`, so the user still cannot see fallback/provider behavior in the editor.
Impact:
- API consumers must handle two provider shapes: top-level metadata for chat and nested metadata elsewhere.
- Some workflows still provide no provider transparency.
- The Definition of Done requires the UI to show actual provider used, but the sidebar does not yet render it.
Recommended fix:
- Pick one response shape and enforce it everywhere. The existing Definition of Done examples use top-level `provider`, `model`, `cost`, and `warnings`.
- Include the same provider fields in all AI success responses and all stream completion events.
- Update the sidebar to render actual provider and fallback warnings in a compact status line near cost.
- Add a static response-contract checklist for every provider-backed route.
### P2: Failed AI Calls Still Usually Do Not Record Failed Cost Attempts
The Definition of Done says failed calls should record an attempt with error status. Many routes still return on provider errors without a cost/status record:
- Clarity check falls back to default questions and returns cost `0` without recording a failed provider attempt in `includes/class-gutenberg-sidebar.php:4054-4071`.
- Regenerate block returns a `regeneration_error` without cost tracking in `includes/class-gutenberg-sidebar.php:3706-3714`.
- Multi-pass refinement returns the provider error directly in `includes/class-gutenberg-sidebar.php:6862-6866`.
- Article refinement returns the provider error directly in `includes/class-gutenberg-sidebar.php:6945-6949`.
Impact:
- Reliability metrics undercount provider failures.
- Users may see no cost but also no durable audit trail explaining why generation failed.
- It becomes harder to debug whether failures are model, provider, prompt, network, or quota related.
Recommended fix:
- Use the same cost helper for failed attempts with `status = 'error'`, `cost = 0`, and available provider/model data.
- Preserve the user-facing error, but write a cost/event row for observability.
### P2: Model Defaults Are Still Fragmented
No central model registry was found. Defaults remain spread across:
- Activation defaults in `wp-agentic-writer.php:140-142`.
- Sidebar defaults in `includes/class-gutenberg-sidebar.php:278-283`.
- Settings defaults and fallbacks in `includes/class-settings.php` and `includes/class-settings-v2.php`.
- OpenRouter provider defaults in `includes/class-openrouter-provider.php:34-69`.
- JavaScript presets in `assets/js/settings-v2.js:35-56`.
- Wrapper fallback model groups in `includes/class-wp-ai-client-wrapper.php:94-100`.
- Image manager fallbacks in `includes/class-image-manager.php:185-249`.
Impact:
- A model default can be changed in one layer while another layer silently keeps the old value.
- Cost estimates, UI presets, activation defaults, and provider runtime defaults can disagree.
Recommended fix:
- Create one PHP model registry for task defaults, labels, capabilities, provider support, and deprecation status.
- Localize JS presets from the registry.
- Add a consistency check that activation/settings/provider defaults match the registry.
### P2: Editor UI/UX Still Needs Browser Verification
The syntax checks are clean, but no live WordPress editor pass was run.
Impact:
- The exact areas still changing, chat hydration, session persistence, provider warnings, streaming completion, and cost display, are frontend workflow problems as much as backend problems.
- Static checks cannot validate `wp.editPost` package availability, REST nonce behavior, editor state persistence, or whether new provider metadata appears to users.
Recommended fix:
- Run a browser pass in the block editor after the next implementation pass.
- Verify sidebar open/persist, chat reload continuity, plan/write/refine cost updates, provider warning display, and unauthorized post failures.
## Recommended Next Work
1. Remove the sidebar dependency on `/chat-history`; load active conversation/session context instead.
2. Replace raw `do_action( 'wp_aw_after_api_request', ... )` calls with one cost helper that always records provider/session/status.
3. Standardize provider metadata response shape and add it to the remaining AI endpoints.
4. Render provider/fallback metadata in the sidebar near cost/status feedback.
5. Change `add_request()` default provider from `openrouter` to `unknown`.
6. Start model registry consolidation after the chat/context and cost contracts are stable.
7. Run the WordPress editor browser workflow pass.
## Current Verdict
The sixth-pass implementation is partially proper. It fixed the migrated session id edge case and the deprecated wrapper's misleading provider default.
It is not audit-clean yet. The biggest remaining risk is now cross-layer consistency: the backend moved toward session/provider/cost contracts, but the sidebar and cost ledger still have old assumptions that can make context history and provider cost reporting disagree with what actually happened.

106
docs/architecture/PLUGIN_AUDIT_RETRACE_SIXTEENTH_PASS_2026-05-26.md Normal file
View File

@@ -0,0 +1,106 @@
# WP Agentic Writer Sixteenth Retrace Audit
Audit date: 2026-05-26
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_FIFTEENTH_PASS_2026-05-26.md`
Scope: sixteenth pass after fifteenth-retrace implementation, covering final static audit state, live editor readiness, provider/cost/context contracts, model preset ownership, and remaining release gate.
## Executive Summary
The fifteenth-pass implementation closed the remaining static cleanup items:
- The duplicate `applyProviderMetadata(data)` call in the normal generation completion branch is removed; the branch now calls it once at `assets/js/sidebar.js:1039-1042`.
- Settings V2 no longer keeps a hard-coded JavaScript preset map; it uses localized PHP presets with an empty fallback at `assets/js/settings-v2.js:32-35`.
- Legacy settings presets are still inline, but are explicitly documented as manually kept in sync with `WP_Agentic_Writer_Settings_V2::get_model_presets()` at `includes/class-settings.php:1025-1028`.
- The cost hook contract remains clean: static scan finds only the central helper hook and the keyword suggester full-contract hook.
- The legacy chat migration P0 remains fixed: no direct `new WP_Agentic_Writer_Context_Service` references were found.
- PHP and JavaScript syntax checks pass.
No new P0, P1, or P2 static implementation defect was found.
The audit chain is now effectively static-clean for the repeatedly retraced areas: chat/context continuity, legacy migration, provider metadata propagation, cost attribution, model registry/default ownership, and syntax. The only remaining gate is live WordPress editor/browser verification.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of fifteenth-pass findings against current code.
- Static scan for short-form `wp_aw_after_api_request` calls.
- Static scan for direct `new WP_Agentic_Writer_Context_Service`.
- Static scan for provider metadata completion branches.
- Static scan for model preset ownership and duplication.
- Static scan for live browser verification evidence.
- No live WordPress editor/browser workflow was run in this pass.
## Fifteenth-Pass Status Trace
| Fifteenth-pass item | Current status | Evidence |
|---|---:|---|
| Live editor/browser verification | Still open | No new verification note or browser evidence was found. |
| Settings V2 JS fallback preset duplication | Fixed | `assets/js/settings-v2.js:35` now falls back to `{}` instead of duplicating the preset map. |
| Legacy settings preset duplication | Accepted/owned | `includes/class-settings.php:1025-1028` says the legacy map is manually kept in sync with Settings V2 presets. |
| Duplicate provider metadata call | Fixed | `assets/js/sidebar.js:1039-1042` contains one `applyProviderMetadata(data)` call before cost update. |
## Static Contract State
### Chat And Context
- Canonical conversation loading remains on `/conversation/{post_id}`.
- Legacy `_wpaw_chat_history` migration uses `WP_Agentic_Writer_Context_Service::get_instance()`.
- No direct construction of `WP_Agentic_Writer_Context_Service` was found.
### Provider Metadata
- Retry chat applies provider metadata on completion.
- Normal stream completion branches apply provider metadata.
- Provider-backed backend responses and stream completions include metadata in the previously retraced paths.
### Cost Tracking
- `includes/class-gutenberg-sidebar.php` routes cost tracking through `track_ai_cost()`.
- `includes/class-keyword-suggester.php` uses the full provider/session/status hook contract.
- Static scan found no short-form seven-argument provider-backed cost hooks.
### Models And Presets
- Registry-backed defaults remain in the active PHP settings/provider paths.
- Settings V2 presets are centralized in `WP_Agentic_Writer_Settings_V2::get_model_presets()`.
- Legacy settings retains an inline preset map, now explicitly marked as manually synchronized legacy behavior.
## Remaining Finding
### P2: Live WordPress Editor/Browser Verification Is Still Required
Static audit is clean, but live editor behavior is still unproven.
Required verification:
- Legacy `_wpaw_chat_history` migrates through `/conversation/{post_id}` without fatal error.
- Sidebar chat persists after editor reload.
- Retry chat updates the provider/fallback badge.
- Provider badge updates after chat, clarity, planning, generation, block refinement, chat refinement, meta, keyword, intent, and improvement actions.
- Cost log rows include provider/session/status for the same actions.
- Model setting changes affect generated requests.
- Unauthorized REST access remains denied.
Impact:
- Without this pass, static code contracts are checked, but WordPress editor UI behavior, persistence, permissions, and rendered state updates are not proven.
Recommended fix:
- Run a live editor verification pass and record the evidence in a short document.
- Suggested document: `docs/architecture/PLUGIN_AUDIT_BROWSER_VERIFICATION_2026-05-26.md`.
- Include post IDs used, actions tested, observed provider badge/cost log behavior, reload behavior, and any screenshots or console/log notes.
## Priority Queue
1. P2: Run live WordPress editor/browser verification and record evidence.
## Completion Criteria For Next Pass
The next retrace can mark this pass complete when:
- A live editor/browser verification note exists for migration, persistence, provider badge updates, cost attribution, model settings, retry chat, and auth denial.
- Any browser-discovered defects are either fixed or moved into a new targeted report.

233
docs/architecture/PLUGIN_AUDIT_RETRACE_SIXTH_PASS_2026-05-25.md Normal file
View File

@@ -0,0 +1,233 @@
# WP Agentic Writer Sixth Retrace Audit
Status: COMPLETE / RETRACED
Completion marker: 2026-05-25
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_SEVENTH_PASS_2026-05-25.md`
Audit date: 2026-05-25
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_FIFTH_PASS_2026-05-25.md`
Scope: sixth pass after fifth-retrace implementation, covering REST authorization, UI/UX risk, conversation context/history, provider transparency, cost attribution, model defaults, and release readiness.
## Executive Summary
The fifth-pass implementation closed the main P0 defects that were causing the "fix A, lose B" cycle:
- Permission checks for the previously flagged post-scoped handlers now happen before post config, meta, content, provider, streaming, or cost work.
- Utility routes that accept `postId` for cost/context attribution now validate `edit_post` before using that post id.
- Legacy chat migration now has stronger migrate-on-read behavior and deletes old `_wpaw_chat_history` after migration.
- PHP and JavaScript syntax checks pass.
I did not find a new P0 blocker in this retrace. The remaining gaps are narrower and mostly about consistency contracts: provider metadata is still only complete in chat, migrate-on-read can still return a stale `session_id`, legacy chat history is still exposed through a deprecated route, model defaults remain fragmented, and browser-level UI compatibility has not been verified.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- Static retrace of fifth-pass findings against current code.
- Static sweep of provider metadata, context migration, cost tracker, model defaults, and legacy chat history surfaces.
- No live WordPress editor browser workflow was run in this pass.
## Fifth-Pass Status Trace
| Fifth-pass item | Current status | Evidence |
|---|---:|---|
| `handle_revise_plan()` permission ordering | Fixed | `postId` is extracted and checked before post config/meta/provider work in `includes/class-gutenberg-sidebar.php:2023-2057`. |
| `handle_block_refine()` permission ordering | Fixed | `edit_post` is checked before post config/provider work in `includes/class-gutenberg-sidebar.php:4207-4236`. |
| `handle_refine_from_chat()` permission ordering | Fixed | `edit_post` is checked before post config/streaming work in `includes/class-gutenberg-sidebar.php:4863-4893`. |
| `handle_generate_meta()` permission ordering | Fixed | `edit_post` is checked before `get_post()` in `includes/class-gutenberg-sidebar.php:6085-6108`. |
| `handle_check_clarity()` permission ordering | Fixed | `edit_post` is checked before resolving post config in `includes/class-gutenberg-sidebar.php:3811-3839`. |
| `handle_summarize_context()` post-id authorization | Fixed | The route validates `edit_post` before provider/cost work in `includes/class-gutenberg-sidebar.php:6266-6278`. |
| `handle_detect_intent()` post-id authorization | Fixed | The route validates `edit_post` before provider/cost work in `includes/class-gutenberg-sidebar.php:6374-6388`. |
| `handle_refine_multi_pass()` post-id authorization | Fixed | The route validates `edit_post` before provider/cost work in `includes/class-gutenberg-sidebar.php:6756-6770`. |
| Migrate-on-read session recovery | Improved, still has one response-identity edge case | `get_context()` uses the migrated id for lookup, but returns the original `$session_id` in the response at `includes/class-context-service.php:72-90`. |
| Provider metadata response contract | Still open | Chat responses include provider transparency; many non-chat responses still omit it. |
| Legacy `/chat-history` endpoint | Still open | The deprecated route remains registered and still returns legacy post meta in `includes/class-gutenberg-sidebar.php:346-354` and `includes/class-gutenberg-sidebar.php:1282-1308`. |
| Deprecated cost wrapper | Still open as low-risk debt | `record_usage()` is deprecated but still hard-codes provider `openrouter` in `includes/class-cost-tracker.php:176-186`. |
| Model registry/default unification | Still open | Defaults remain spread across activation, settings, providers, JS presets, wrappers, and image manager. |
| Sidebar browser compatibility | Unverified | Syntax passes, but no WordPress editor/browser pass was run. |
## Remaining Findings
### P1: Non-Chat AI Responses Still Do Not Expose Provider Transparency
Chat now satisfies the provider transparency contract by returning `provider`, `selected_provider`, `fallback_used`, and `warnings` in normal and streaming completion paths at `includes/class-gutenberg-sidebar.php:1049-1053` and `includes/class-gutenberg-sidebar.php:1220-1227`.
The same contract is still missing from many non-chat AI endpoints:
- Plan generation tracks the actual provider but returns only `plan`, `cost`, and `web_search_results` at `includes/class-gutenberg-sidebar.php:1992-2013`.
- Plan revision tracks the actual provider but returns only `plan` and `cost` at `includes/class-gutenberg-sidebar.php:2153-2172`.
- Block refinement returns `blocks`, `blockId`, and `cost` only at `includes/class-gutenberg-sidebar.php:4338-4355`.
- Multi-pass refinement returns `pass`, `refined_content`, and `cost` only at `includes/class-gutenberg-sidebar.php:6808-6825`.
Impact:
- Users and support logs can see provider/fallback behavior for chat, but not for planning, writing, refinement, clarity, SEO/meta, or utility AI actions.
- When fallback routing or local/cloud routing differs by task, the UI cannot explain why costs, quality, or latency changed.
- Cost records may be accurate internally while the user-visible response stays opaque.
Recommended fix:
- Add one shared response metadata helper, for example `build_provider_metadata( $provider_result )`.
- Add the same keys to every non-streaming AI response:
- `provider`
- `selected_provider`
- `fallback_used`
- `warnings`
- Add equivalent completion metadata to every streaming AI action, not only chat.
- Add focused REST response tests for plan, revise, block refine, refine multi-pass, clarity, and meta generation.
### P1: Migrated Context Can Return Messages From One Session But Report Another Session ID
`get_context()` now does the important migration recovery step: when the requested session does not exist and legacy post history exists, it migrates the legacy history and then fetches the effective migrated session id.
The remaining problem is the response identity:
- Migration returns `$migrated_session_id` at `includes/class-context-service.php:72`.
- `$effective_session_id` is used to fetch `$session` at `includes/class-context-service.php:74-75`.
- The returned context still uses the original `$session_id` at `includes/class-context-service.php:89-90`.
Impact:
- A client can receive migrated messages/context but keep using an empty or stale session id.
- That can fragment follow-up chat into a different session, making "conversation memory disappeared" bugs look random.
- This is especially risky during migration from legacy `_wpaw_chat_history` to conversation sessions because it happens only for older posts.
Recommended fix:
- After successful migration lookup, set `$session_id = $effective_session_id` before returning context.
- Prefer returning `$session['session_id'] ?? $effective_session_id` if the session object carries its canonical id.
- Add a migration test where `get_context( 'missing-session', $post_id )` creates a new session and verifies that the response `session_id` equals the migrated session id.
### P2: Deprecated `/chat-history` Still Exposes Legacy Post-Meta History
The legacy route is still registered:
- `GET /wp-agentic-writer/v1/chat-history/(?P<post_id>\d+)` remains active at `includes/class-gutenberg-sidebar.php:346-354`.
- `handle_get_chat_history()` still returns values from `_wpaw_chat_history` with a `deprecated` marker at `includes/class-gutenberg-sidebar.php:1282-1308`.
- `get_post_chat_history()` still reads `_wpaw_chat_history` directly at `includes/class-gutenberg-sidebar.php:1336-1346`.
Impact:
- This is not an immediate permission issue because the route checks `edit_post`.
- It is still product debt because there are now two observable history APIs: legacy post meta and conversation sessions.
- UI or integrations can keep depending on the old endpoint and bypass the new context/session model.
Recommended fix:
- Replace the response body with a conversation-session-backed compatibility payload, or return a structured `410`/deprecation response after one release window.
- Track any client code still calling `/chat-history` before removal.
- Add a regression test proving no active UI path uses this endpoint.
### P2: Deprecated Cost Wrapper Still Defaults Provider To OpenRouter
`record_usage()` is now explicitly deprecated, which is good. It still calls `record_usage_full()` with provider `openrouter` at `includes/class-cost-tracker.php:176-186`.
Impact:
- Any remaining caller using the legacy wrapper can create misleading provider attribution.
- This is lower risk than before because newer paths use provider metadata, but it can still pollute reporting.
Recommended fix:
- Change the default provider to `legacy_unknown` or `unknown`.
- Log a debug warning when this wrapper is called so remaining callers can be eliminated.
- Search for all direct calls to `record_usage()` and migrate them to `record_usage_full()`.
### P2: Model Defaults Are Still Fragmented Across Runtime Surfaces
Model defaults are still declared in multiple places with conflicting generations and ids:
- Activation defaults in `wp-agentic-writer.php:140-142`.
- Sidebar defaults in `includes/class-gutenberg-sidebar.php:276-283`.
- Settings defaults and sanitization fallbacks in `includes/class-settings.php` and `includes/class-settings-v2.php`.
- Provider property defaults in `includes/class-openrouter-provider.php:34-69`.
- JavaScript presets in `assets/js/settings-v2.js:35-56`.
- Wrapper fallback groups in `includes/class-wp-ai-client-wrapper.php:94-100`.
- Image-manager fallbacks in `includes/class-image-manager.php:185-249`.
Impact:
- Changing a default model in one layer can silently diverge from another layer.
- UI presets, provider runtime defaults, activation defaults, and cost estimation can disagree.
- This is exactly the kind of area where fixing A can regress B.
Recommended fix:
- Create one PHP model registry for task defaults, labels, capabilities, pricing hints, provider support, and deprecation status.
- Generate or localize the JS presets from that registry instead of hard-coding them independently.
- Add a consistency test that checks all task defaults exist in the registry and that activation/settings/provider fallbacks match it.
### P2: Sidebar UI Needs A Real WordPress Editor Compatibility Pass
`assets/js/sidebar.js` and `assets/js/settings-v2.js` both pass syntax checks, but syntax is not enough for the editor UI.
Impact:
- WordPress package availability, `wp.editPost.PluginSidebar` compatibility, REST nonce behavior, streaming UI states, and cost display states can still fail only inside the block editor.
- The plugin has many user-facing states now: chat, planning, refinement, clarity, model routing, cost display, warnings, and session history. A static pass cannot validate their interaction quality.
Recommended fix:
- Run a browser pass inside the WordPress editor with the plugin enabled.
- Cover at minimum:
- Sidebar opens and persists.
- Chat session continues after reload.
- Provider/fallback warnings render where metadata exists.
- Cost display updates after chat, plan, refine, and meta-generation actions.
- Unauthorized post access fails cleanly without partial UI mutation.
- Model settings changes are reflected in generated requests.
## System-Level Opportunities
### 1. Add A Shared Guard For Post-Scoped REST Requests
The fifth-pass fixes were successful, but they were route-by-route. To keep this from regressing, add a helper such as:
```php
private function require_post_permission_from_request( WP_REST_Request $request, $field = 'postId' ) {
$post_id = absint( $request->get_param( $field ) );
if ( $post_id > 0 && ! $this->check_post_permission( $post_id ) ) {
return new WP_Error( 'forbidden', __( 'You do not have permission to access this post.', 'wp-agentic-writer' ), array( 'status' => 403 ) );
}
return $post_id;
}
```
Then use it immediately after request parsing in every route that reads, writes, streams, or attributes cost to a post.
### 2. Standardize AI Response Envelopes
The plugin needs one response shape for all AI actions:
```json
{
"data": {},
"cost": 0,
"provider": "openrouter",
"selected_provider": "openrouter",
"fallback_used": false,
"warnings": []
}
```
The exact shape can differ, but the metadata keys should not. This will make UI, debugging, and cost review dramatically simpler.
### 3. Treat Conversation Session ID As A Canonical Contract
The remaining migration edge case is a sign that `session_id` should be treated as a canonical response contract. Any endpoint that creates, migrates, clears, or resumes a session should return the active session id and the UI should update local state from that value.
## Recommended Next Work
1. Fix `get_context()` so migrated reads return the effective session id.
2. Add provider metadata to every non-chat AI response and stream completion event.
3. Replace or retire `/chat-history` compatibility behavior.
4. Change deprecated cost wrapper provider attribution from `openrouter` to `legacy_unknown`.
5. Start model registry consolidation.
6. Run the WordPress editor browser pass before calling the audit chain fully closed.
## Current Verdict
The fifth-pass implementation is proper for the critical authorization and cost-attribution issues it targeted. I would mark the fifth-pass P0 items complete.
The plugin is not fully audit-clean yet. The next highest-value work is now chat/context correctness plus response transparency: fix the migrated `session_id` edge case and make provider/cost metadata consistent across every AI action.

175
docs/architecture/PLUGIN_AUDIT_RETRACE_TENTH_PASS_2026-05-26.md Normal file
View File

@@ -0,0 +1,175 @@
# WP Agentic Writer Tenth Retrace Audit
Status: COMPLETE / RETRACED
Completion marker: 2026-05-26
Follow-up report: `docs/architecture/PLUGIN_AUDIT_RETRACE_ELEVENTH_PASS_2026-05-26.md`
Audit date: 2026-05-26
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_NINTH_PASS_2026-05-26.md`
Scope: tenth pass after ninth-retrace implementation, covering failed-attempt tracking, provider transparency, model registry adoption, chat/context compatibility, cost tracking contracts, UI/UX, and release readiness.
## Executive Summary
The ninth-pass implementation fixed the P0 runtime blocker:
- The failed-attempt branches no longer call `$provider_result->get_default_model()`.
- They now use `WPAW_Model_Registry::get_default_model()` for writing, clarity, and refinement paths.
- PHP and JavaScript syntax checks pass.
The implementation also improved two other areas:
- Provider metadata now reaches the sidebar in the streaming completion path and is rendered as a compact provider/fallback badge near cost.
- Model registry adoption improved across active settings paths, sidebar defaults, activation defaults, image-manager analysis/prompt paths, and failed-attempt model fallbacks.
No new P0 blocker was found in this retrace. The remaining gaps are now mostly completion and verification work:
- Provider metadata UI is only wired for the streaming completion path, not every non-streaming AI response path.
- The model registry exists and is partially adopted, but some model defaults/fallbacks are still hard-coded.
- The sidebar still hydrates chat from the deprecated `/chat-history` compatibility route.
- Raw cost hook calls still exist outside `track_ai_cost()`.
- Live WordPress editor browser verification is still pending.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of ninth-pass findings against current code.
- Static sweep of failed-attempt tracking, provider metadata UI usage, model registry adoption, chat-history usage, and raw cost hooks.
- No live WordPress editor/browser workflow was run in this pass.
## Ninth-Pass Status Trace
| Ninth-pass item | Current status | Evidence |
|---|---:|---|
| P0 failed-attempt fatal from `$provider_result->get_default_model()` | Fixed | Failed branches now call `WPAW_Model_Registry::get_default_model()` at `includes/class-gutenberg-sidebar.php:3796-3805`, `4163-4172`, `7004-7012`, and `7102-7110`. |
| Failed-attempt cost helper | Improved | `track_ai_cost()` exists at `includes/class-gutenberg-sidebar.php:966-1005` and is used in sampled failure branches. |
| Provider metadata backend envelope | Improved | Chat adds `provider_metadata` at `includes/class-gutenberg-sidebar.php:1114`, while many backend responses already include the same envelope. |
| Provider metadata editor rendering | Partially fixed | Sidebar captures provider metadata from streaming `complete` events at `assets/js/sidebar.js:1023-1031` and renders a provider/fallback badge at `assets/js/sidebar.js:4658-4665` and `assets/js/sidebar.js:4701-4708`. |
| Model registry adoption | Improved but incomplete | Active settings defaults use `WPAW_Model_Registry` in `includes/class-settings-v2.php:100-111`, `989-994`, and `1109-1114`; hard-coded defaults remain elsewhere. |
| Sidebar dependency on `/chat-history` | Still open | `assets/js/sidebar.js:644-668` still fetches `/chat-history/${postId}`. |
| Raw cost hook drift | Still open | Many direct `do_action( 'wp_aw_after_api_request', ... )` calls remain in `includes/class-gutenberg-sidebar.php`. |
| WordPress editor browser pass | Still open | Syntax checks passed, but no live editor workflow was verified. |
## Remaining Findings
### P1: Provider Metadata UI Is Only Partially Wired
The sidebar now has provider metadata state and renders it:
- `providerInfo` state exists at `assets/js/sidebar.js:73`.
- Streaming completion events capture `data.provider` or `data.provider_metadata` at `assets/js/sidebar.js:1023-1031`.
- A provider/fallback badge renders near cost in the focus keyword UI at `assets/js/sidebar.js:4658-4665` and `assets/js/sidebar.js:4701-4708`.
The gap is coverage. The only `setProviderInfo()` call found is in the streaming completion handler. Many non-streaming fetch flows parse JSON responses that may include `provider_metadata`, but they do not appear to update `providerInfo`:
- Plan generation/revision non-stream paths.
- Chat non-stream responses if enabled.
- Meta generation.
- Summarize context and intent detection.
- Refine/reformat utility calls.
Impact:
- Provider transparency is visible only for some workflows.
- Users can still run AI actions with provider metadata in the response but no visible provider/fallback update in the sidebar.
- The Definition of Done expects actual provider visibility across AI actions, not only one streaming flow.
Recommended fix:
- Add one frontend helper, for example `applyProviderMetadata(responseData)`, and call it after every AI JSON response and stream completion.
- Support both `provider_metadata` and top-level provider fields while backend response shapes continue to coexist.
- Add a small UI checklist or browser assertion that provider info updates after chat, plan, refine, meta, and utility actions.
### P1: Model Registry Adoption Is Improved But Still Not A Single Source Of Truth
The registry is now more than a stub:
- `WPAW_Model_Registry` exists in `includes/class-model-registry.php`.
- Activation uses registry defaults at `wp-agentic-writer.php:140-145`.
- Active settings V2 localization and sanitization use registry defaults at `includes/class-settings-v2.php:100-111` and `includes/class-settings-v2.php:989-994`.
- Sidebar defaults use registry defaults at `includes/class-gutenberg-sidebar.php:278-283`.
- Image-manager analysis/prompt paths now use registry defaults at `includes/class-image-manager.php:183-249`.
But several hard-coded model defaults or model lists remain:
- `includes/class-settings-v2.php:188-215` still has fallback model arrays with literal model ids.
- `includes/class-settings-v2.php:224-230` still uses literal fallback ids in model transformation.
- `assets/js/settings-v2.js:32-58` still hard-codes budget/balanced/premium preset ids.
- `includes/class-openrouter-provider.php:29-75` still hard-codes provider property defaults, even though comments say they are registry-sourced.
- `includes/class-image-manager.php:409-478` still hard-codes image-model fallback values.
- Legacy `includes/class-settings.php` still contains old hard-coded defaults, and `wp-agentic-writer.php:101-104` can still instantiate it if Settings V2 is unavailable.
Impact:
- Runtime defaults can still drift from registry defaults.
- Fallback UI/model lists can disagree with actual generation defaults.
- The registry is useful now, but it is not yet enforcing the "single source of truth" claim.
Recommended fix:
- Replace remaining fallback literals in active runtime paths with `WPAW_Model_Registry::get_default_model()` or `get_fallback_model()`.
- Decide whether JS presets are curated product presets or registry-derived defaults. If curated, document them outside the "single source of truth" claim.
- Initialize OpenRouter default properties from the registry in the constructor, or remove property defaults as authoritative values.
- Either update legacy `class-settings.php` or clearly mark it as inactive/deprecated and remove fallback instantiation.
- Add a static check that flags task-default strings outside the registry, except approved curated presets and pricing maps.
### P2: Sidebar Still Depends On Deprecated `/chat-history`
The earlier data-loss bug is fixed, but the frontend still hydrates chat through a deprecated compatibility endpoint:
- `assets/js/sidebar.js:644-668` calls `/chat-history/${postId}`.
- The backend compatibility route remains registered at `includes/class-gutenberg-sidebar.php:346-354`.
Impact:
- The UI still depends on a route whose name and docblock communicate legacy post-meta history.
- Future cleanup could accidentally break chat hydration.
- It keeps one old mental model alive even though conversations are now session-backed.
Recommended fix:
- Move sidebar hydration to the canonical conversation/session context endpoint.
- If `/chat-history` remains, update its docblock and response contract to make clear that it returns session-backed compatibility data.
### P2: Raw Cost Hook Calls Still Bypass The New Helper
The new `track_ai_cost()` helper is a good step, but direct `do_action( 'wp_aw_after_api_request', ... )` calls remain throughout `includes/class-gutenberg-sidebar.php`.
Impact:
- New work can still reintroduce incomplete provider/session/status tracking.
- Cost tracking consistency still depends on manual discipline.
Recommended fix:
- Convert direct cost hook calls in `class-gutenberg-sidebar.php` to `track_ai_cost()`.
- Add a static guard that only allows raw `wp_aw_after_api_request` in `track_ai_cost()` and the cost tracker registration.
### P2: Live Editor Browser Verification Still Remains
The screenshot correctly identifies browser verification as the remaining manual task. Static checks cannot prove the editor workflow works.
Recommended browser checklist:
- Sidebar opens and persists in the block editor.
- Chat session continues after page reload.
- Provider/fallback warnings render when metadata exists.
- Cost display updates after chat, plan, refine, and meta actions.
- Unauthorized post access fails cleanly.
- Model settings changes reflect in generated requests.
## Recommended Next Work
1. Add a shared frontend `applyProviderMetadata()` helper and call it for every AI response path.
2. Finish model registry adoption in active runtime paths and document any intentionally curated model presets.
3. Move sidebar chat hydration off `/chat-history`, or update that compatibility endpoint's contract and docblock.
4. Convert remaining raw cost hooks to `track_ai_cost()` and add a static guard.
5. Run the live WordPress editor browser workflow pass.
## Current Verdict
The ninth-pass implementation is substantially better and fixes the P0 runtime fatal. I would mark the ninth-pass blocker complete.
The plugin is close to leaving the audit-chain loop, but not fully closed. The remaining work is mostly consistency and verification: complete provider metadata UI coverage, finish model registry adoption, remove or formalize the deprecated chat-history dependency, and run the live editor pass.

249
docs/architecture/PLUGIN_AUDIT_RETRACE_THIRD_PASS_2026-05-24.md Normal file
View File

@@ -0,0 +1,249 @@
# WP Agentic Writer Third Retrace Audit
Status: COMPLETE / SUPERSEDED
Completion marker date: 2026-05-25
Next retrace report: `docs/architecture/PLUGIN_AUDIT_RETRACE_FOURTH_PASS_2026-05-25.md`
Audit date: 2026-05-24
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_SECOND_PASS_2026-05-24.md`
Scope: third pass after second-retrace implementation, covering UI/UX, editor runtime, REST authorization, conversation context/history, cost tracking, provider/model metadata, migrations, and release readiness.
## Executive Summary
The second-pass implementation closed several concrete blockers:
- `assets/js/sidebar.js` now passes JavaScript syntax validation.
- `assets/js/settings-v2.js` still passes JavaScript syntax validation.
- Full PHP syntax validation passes.
- `WP_Agentic_Writer_Cost_Tracker::record_usage()` now exists, so the previous missing-method fatal is closed.
- Conversation migration now reads `wpaw_conversations_db_version`.
- Deactivation now clears `wpaw_cleanup_old_sessions`.
- Previously named post-config, cost, section-block, and image REST handlers now check target-post permissions.
The plugin is still not ready to move exclusively into new chat/context feature work. The largest remaining risks are now more subtle:
1. The sidebar debug logger is syntactically valid but recursively calls itself, so any error log path can crash into a stack overflow.
2. Core generation/chat REST endpoints still accept `postId` under generic `edit_posts` permission and then read or write target post meta.
3. Clear context accepts `sessionId`, but the current frontend calls do not send one, and the sidebar no longer appears to maintain a session id in state.
4. The legacy chat-history route and legacy post-meta helper methods remain active, while the context service comment says legacy history should migrate on read.
5. Cost compatibility tracking no longer fatals, but it records misleading provider/model metadata.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- Static trace of second-pass findings against current code.
- No live WordPress browser workflow was run in this pass.
## Second-Pass Status Trace
| Second-pass item | Current status | Evidence |
|---|---:|---|
| Sidebar JS parse failure | Fixed, but runtime logger bug remains | `node -c assets/js/sidebar.js` passes; logger recursively calls itself at `assets/js/sidebar.js:17-20`. |
| Missing `record_usage()` method | Fixed, but attribution is inaccurate | Method exists at `includes/class-cost-tracker.php:164-176`; wrapper still passes provider names through the `$model` parameter. |
| Conversation migration version gate | Improved | `wpaw_run_migrations()` reads `wpaw_conversations_db_version` at `includes/class-conversation-migration.php:67-72`; main table bootstrap also checks it at `wp-agentic-writer.php:250-259`. |
| Clear-context uses context service | Partially fixed | Handler accepts `sessionId` and calls context service at `includes/class-gutenberg-sidebar.php:1230-1244`; frontend clear calls omit `sessionId`. |
| Post-config/cost/section/image permissions | Fixed for named handlers | Target post checks added in the handlers traced below. |
| Cost table base schema | Improved | Base schema includes `session_id`, `provider`, and `status`; runtime upgrade still assumes the table exists. |
| Deactivation cleanup | Fixed | `wp_clear_scheduled_hook( 'wpaw_cleanup_old_sessions' )` added at `wp-agentic-writer.php:271-278`. |
| Model registry/default unification | Still open | Defaults remain spread across activation, settings PHP, settings JS, providers, image manager, and WP AI wrapper. |
## Critical Findings
### P0: Sidebar Logger Recurses Instead of Calling Console
The syntax error from the last report is fixed, but the logger implementation is still broken:
- `assets/js/sidebar.js:17` defines `log` as `wpawLog.log('[WPAW]', ...args)`.
- `assets/js/sidebar.js:18` defines `error` as `wpawLog.error('[WPAW]', ...args)`.
- `assets/js/sidebar.js:20` defines `warn` as `wpawLog.warn('[WPAW]', ...args)`.
Impact:
- Any `wpawLog.error()` call can recurse until the browser throws a maximum call stack error.
- With debug enabled, any `wpawLog.log()` or `wpawLog.warn()` call can do the same.
- Error handling becomes unreliable exactly when the sidebar needs it most, especially during streaming, generation, migration, and cost refresh failures.
Recommended fix:
- Change the sidebar logger to match the working settings logger:
- `log: (...args) => { if (isDebug) console.log('[WPAW]', ...args); }`
- `error: (...args) => console.error('[WPAW]', ...args)`
- `info: (...args) => { if (isDebug) console.info('[WPAW]', ...args); }`
- `warn: (...args) => { if (isDebug) console.warn('[WPAW]', ...args); }`
- Add a simple static check or unit test that rejects `wpawLog.*` inside the logger object itself.
### P0: Core Post-Scoped Generation Routes Still Lack Upfront `edit_post` Checks
The named endpoints from the previous report were patched, but the larger route family still relies on generic `edit_posts` route permission:
- `/chat` is registered with only `check_permissions` at `includes/class-gutenberg-sidebar.php:325-333`.
- `/generate-plan` is registered with only `check_permissions` at `includes/class-gutenberg-sidebar.php:377-385`.
- `/execute-article` is registered with only `check_permissions` at `includes/class-gutenberg-sidebar.php:398-406`.
- `check_permissions()` only checks `current_user_can( 'edit_posts' )` at `includes/class-gutenberg-sidebar.php:930-932`.
Handlers then use `postId` to read or mutate post-scoped data:
- `handle_chat_request()` reads post config, detected language, and focus keyword from the target post at `includes/class-gutenberg-sidebar.php:955-978`.
- `handle_generate_plan()` reads post memory and writes `_wpaw_plan`, `_wpaw_detected_language`, and `_wpaw_memory` at `includes/class-gutenberg-sidebar.php:1825-1969`.
- `handle_execute_article()` reads `_wpaw_plan` and can update `_wpaw_plan` after generation at `includes/class-gutenberg-sidebar.php:2925-3176`.
- `handle_reformat_blocks()` reads `_wpaw_plan` at `includes/class-gutenberg-sidebar.php:3509-3529`.
- `handle_regenerate_block()` accepts `postId` and records cost against it at `includes/class-gutenberg-sidebar.php:3594-3642`.
Impact:
- A user with generic author/editor access can potentially read or affect plugin state for posts they cannot edit.
- Cost records can be attributed to unauthorized posts.
- Chat/context behavior can leak focus keywords, detected language, memory summaries, and plans across post boundaries.
Recommended fix:
- Add an upfront helper such as `require_post_permission_from_params( $params, 'postId' )`.
- Call it in every handler that accepts `postId` or reads/writes post meta, not only the endpoints listed in a prior audit.
- For postless conversation mode, allow `postId = 0` only when no post meta is read or written.
## High Priority Findings
### P1: Clear Context Still Does Not Clear the Active Session From the UI
The backend handler now accepts `sessionId` and calls the context service:
- `handle_clear_context()` reads `sessionId` at `includes/class-gutenberg-sidebar.php:1230-1234`.
- It calls `$this->context_service->clear_context( $session_id, $post_id )` at `includes/class-gutenberg-sidebar.php:1243-1244`.
But both frontend clear calls still send only the post id:
- Reset command sends `JSON.stringify({ postId: postId })` at `assets/js/sidebar.js:1661-1669`.
- Clear chat context sends `JSON.stringify({ postId })` at `assets/js/sidebar.js:2023-2031`.
- A search of `assets/js/sidebar.js` found no active `sessionId`/`currentSession` state being maintained.
Impact:
- Clear context deletes legacy post meta but does not clear the active conversation table row when the session id is omitted.
- Users can see a cleared frontend, then later reload or resume a session that still contains old messages.
- This undermines the goal of making the conversations table the source of truth.
Recommended fix:
- Reintroduce explicit active session state in the sidebar.
- Send `sessionId` on `/clear-context`.
- If the UI cannot provide a session id, the backend should clear active sessions for that post owned by the current user or return a clear error explaining that the session id is required.
### P1: Legacy Chat History Read/Write Helpers Still Exist Beside the Session Store
The main chat send path no longer writes `_wpaw_chat_history`, but old route/helper code remains:
- `/chat-history/(?P<post_id>\d+)` is still registered at `includes/class-gutenberg-sidebar.php:346-354`.
- `handle_get_chat_history()` reads legacy post meta at `includes/class-gutenberg-sidebar.php:1261-1277`.
- `update_post_chat_history()` still writes `_wpaw_chat_history` at `includes/class-gutenberg-sidebar.php:1289-1322`.
- `migrate_legacy_chat_history()` still keeps `_wpaw_chat_history` after migration because deletion is commented out at `includes/class-context-service.php:299-300`.
- The context service header says legacy history migrates on read, but `get_context()` does not call migration when no session exists at `includes/class-context-service.php:62-87`.
Impact:
- The system still has two history stores and one migration route, rather than one reliable source of truth.
- Legacy meta can be migrated repeatedly or surfaced by old endpoints after the active session has diverged.
- Future chat/context work remains likely to regress because old and new paths coexist.
Recommended fix:
- Remove or deprecate the legacy `/chat-history` endpoint once the frontend uses conversation sessions.
- Delete legacy post meta after successful migration, or write a `_wpaw_chat_history_migrated` marker and never re-import it.
- Make `get_context()` perform migrate-on-read if the post has legacy history and no session exists.
### P1: Cost Compatibility Method Prevents Fatal, But Mislabels Model/Provider
`record_usage()` now exists, but its signature and callers do not align with the newer cost schema:
- Method signature is `record_usage( $post_id, $action, $model, $cost, $session_id = '' )` at `includes/class-cost-tracker.php:164`.
- It always records provider as `'openrouter'` at `includes/class-cost-tracker.php:172`.
- The WP AI core path passes `'core'` as the `$model` argument at `includes/class-wp-ai-client-wrapper.php:197-202`.
- The legacy provider path passes `$provider_result->actual_provider` as the `$model` argument at `includes/class-wp-ai-client-wrapper.php:249-254`.
Impact:
- Cost rows from WP AI wrapper paths can show model=`core` or model=`local_backend/openrouter` and provider=`openrouter`, regardless of the actual provider/model.
- Cost analytics, provider comparison, and budget debugging become unreliable.
Recommended fix:
- Change `record_usage()` to accept an options array or explicit `$provider`, `$model`, `$input_tokens`, `$output_tokens`, `$session_id`, `$status`.
- Update wrapper callers to pass the selected model and actual provider separately.
- Keep the old positional method only as a deprecated compatibility wrapper.
### P1: Cost Table Runtime Upgrade Still Does Not Self-Heal a Missing Table
The base schema is improved, but runtime upgrade remains fragile:
- `maybe_upgrade_table()` calls `DESCRIBE {$table_name}` at `includes/class-cost-tracker.php:71-76`.
- It does not check whether the table exists before calling `in_array()` on the column list.
- Main table creation can still be skipped when `wpaw_db_version` is current at `wp-agentic-writer.php:230-260`.
Impact:
- A site with a current main DB version but missing cost table may not recover cleanly.
- First access to the cost tracker can warn or fail before any cost row is recorded.
Recommended fix:
- Add a `SHOW TABLES LIKE` guard before `DESCRIBE`.
- If the table is missing, call the idempotent cost table creator.
- Prefer per-table schema checks over relying only on `wpaw_db_version`.
## Medium Priority Findings
### P2: Provider Metadata Is Still Incomplete Outside Chat
Chat responses and streaming completion events include provider metadata, but several non-chat routes still return only generated content, variants, or blocks. Cost records often receive provider metadata through the hook, but the UI/API response contract is not uniform.
Recommended fix:
- Standardize generated response envelopes across plan, execute, refine, keyword, image recommendation, and image variant endpoints.
- Include `provider`, `selected_provider`, `fallback_used`, `warnings`, `model`, and `cost` where available.
### P2: Model Defaults Remain Fragmented
Defaults still exist in multiple places:
- Activation defaults in `wp-agentic-writer.php`.
- Sidebar localized defaults in `includes/class-gutenberg-sidebar.php`.
- Settings V1/V2 PHP defaults in `includes/class-settings.php` and `includes/class-settings-v2.php`.
- JS presets in `assets/js/settings-v2.js`.
- Provider defaults in `includes/class-openrouter-provider.php`, `includes/class-local-backend-provider.php`, `includes/class-codex-provider.php`, and `includes/class-wp-ai-client-wrapper.php`.
- Image defaults in `includes/class-image-manager.php`.
Impact:
- A model update can appear fixed in settings but remain stale in provider/runtime paths.
- Cost estimates, UI presets, and actual generation model can diverge.
Recommended fix:
- Create a PHP model registry/default resolver and localize its resolved values to JS.
- Keep legacy keys such as `execution_model` only as migrations into canonical task keys.
### P2: Sidebar Uses `wp.editPost` Without the Previous Fallback
The current sidebar imports `PluginSidebar` from `wp.editPost` at `assets/js/sidebar.js:8-10`. The previous code had a fallback for `wp.editor` versus `wp.editPost`. This may be intentional, but it should be verified against the WordPress versions the plugin supports.
Recommended fix:
- Browser-test the sidebar on the minimum supported WordPress version.
- Restore a compatibility fallback if needed.
## Definition of Done Gates for This Pass
Before considering this retrace implementation complete:
- Fix sidebar logger recursion and verify error/debug calls in browser devtools.
- Add target-post permission checks to all handlers that accept `postId`, especially chat, generate-plan, execute-article, reformat-blocks, regenerate-block, clarity/SEO/GEO, refinement, and any post-scoped streaming path.
- Make clear-context clear the actual active session from the UI, not only legacy post meta.
- Remove, migrate, or hard-deprecate legacy `_wpaw_chat_history` routes and helper writes.
- Fix `record_usage()` metadata attribution so model and provider are not swapped or hardcoded.
- Add a missing-table guard to the cost tracker migration path.
- Keep PHP and JS syntax checks passing.
## Current Decision
Do not move fully into new chat/context implementation yet. The second-pass checklist is mostly implemented, but the current code still has one sidebar runtime blocker, one broad post-authorization gap, and an incomplete conversation source-of-truth transition. Fix those first, then chat/context work will have a much firmer base.

178
docs/architecture/PLUGIN_AUDIT_RETRACE_THIRTEENTH_PASS_2026-05-26.md Normal file
View File

@@ -0,0 +1,178 @@
# WP Agentic Writer Thirteenth Retrace Audit
Audit date: 2026-05-26
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_TWELFTH_PASS_2026-05-26.md`
Scope: thirteenth pass after twelfth-retrace implementation, covering legacy chat migration, conversation context continuity, cost attribution, provider metadata coverage, model registry ownership, UI/UX readiness, and release verification.
Status: COMPLETE / RETRACED
Completion marker: 2026-05-26
Follow-up retrace: `docs/architecture/PLUGIN_AUDIT_RETRACE_FOURTEENTH_PASS_2026-05-26.md`
> This thirteenth-pass report has been implemented and retraced. Keep this document as historical evidence only; use the fourteenth-pass report for current remaining work.
## Executive Summary
The twelfth-pass implementation closed several items from the previous report:
- The canonical `/conversation/{post_id}` handler now attempts legacy `_wpaw_chat_history` migration.
- The specific keyword and improvement suggestion cost hooks called out in the twelfth report now pass the full hook contract.
- The missed frontend provider metadata calls at the previously listed generation/clarity branches were added.
- The deprecated chat-history docblock now describes migration behavior accurately.
- Backup JavaScript endpoint references were removed from `assets/`.
However, a new P0 runtime regression was found in the legacy migration implementation: the new `/conversation/{post_id}` migration branch directly instantiates `WP_Agentic_Writer_Context_Service`, but that service has a private constructor. This will fatal when a legacy post has `_wpaw_chat_history` and no existing conversation session.
Static syntax checks still pass, but PHP linting cannot detect this runtime visibility error.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of twelfth-pass findings against current code.
- Static sweep of legacy chat migration, provider metadata propagation, raw cost hooks, model default ownership, stale backup files, and browser-verification evidence.
- No live WordPress editor/browser workflow was run in this pass.
## Twelfth-Pass Status Trace
| Twelfth-pass item | Current status | Evidence |
|---|---:|---|
| `/conversation/{post_id}` should migrate legacy chat history | Implemented, but broken by P0 | Migration branch exists at `includes/class-gutenberg-sidebar.php:1411-1432`, but calls a private constructor at line `1415`. |
| Keyword suggestion cost attribution | Fixed for called-out path | `includes/class-keyword-suggester.php:140-151` now passes provider/session/status fields. |
| Improvement suggestion cost attribution | Fixed for called-out path | `includes/class-gutenberg-sidebar.php:6911-6922` now passes provider/session/status fields. |
| Provider metadata calls for listed frontend branches | Mostly fixed | `applyProviderMetadata()` is now present at the previously missed branches, including `assets/js/sidebar.js:3698`, `3932`, and `4821`. |
| Model preset/default ownership | Partially de-scoped | JS and legacy presets now document that they are curated, but duplicated hard-coded presets remain active. |
| Browser verification | Still open | Static checks passed; no live editor workflow evidence was found. |
| Stale chat-history comments and backup files | Fixed | Docblock updated at `includes/class-gutenberg-sidebar.php:1346-1350`; no `*.bak` or `*.backup` files remain under `assets/`. |
## Remaining Findings
### P0: Legacy Chat Migration Path Can Fatal Due Private Singleton Constructor
The twelfth implementation added the needed migrate-on-read behavior to the canonical conversation endpoint:
- `includes/class-gutenberg-sidebar.php:1411-1432` checks `_wpaw_chat_history`, attempts migration, then returns the newly created session.
But the migration branch instantiates the context service directly:
- `includes/class-gutenberg-sidebar.php:1415` uses `new WP_Agentic_Writer_Context_Service()`.
That class is explicitly a singleton:
- `includes/class-context-service.php:41-45` exposes `WP_Agentic_Writer_Context_Service::get_instance()`.
- `includes/class-context-service.php:51-53` declares `private function __construct()`.
Impact:
- A legacy post with `_wpaw_chat_history` and no existing conversation session can trigger a fatal error when the editor loads chat through `/conversation/{post_id}`.
- This is exactly the legacy continuity path the twelfth pass intended to repair.
- PHP syntax checks still pass because this is a runtime visibility error, not a parse error.
Recommended fix:
- Replace `new WP_Agentic_Writer_Context_Service()` with `WP_Agentic_Writer_Context_Service::get_instance()`.
- After migration, prefer the returned `$migrated_session_id` when fetching/returning the session, with a fallback to `get_session_by_post_id()`.
- Add a regression test or manual fixture for: post has `_wpaw_chat_history`, has no conversation row, editor loads `/conversation/{post_id}`.
### P1: Raw Cost Hook Drift Still Leaves Provider Attribution Gaps
The two cost examples from the twelfth report were fixed, but a broader scan still found multiple seven-argument `wp_aw_after_api_request` calls in `includes/class-gutenberg-sidebar.php`. These calls still omit provider, session id, and status.
Examples:
- Section execution cost at `includes/class-gutenberg-sidebar.php:3034-3042`.
- Write-from-outline section cost at `includes/class-gutenberg-sidebar.php:3675-3683`.
- Block refinement cost at `includes/class-gutenberg-sidebar.php:4590-4598`.
- Streaming block refinement cost at `includes/class-gutenberg-sidebar.php:4718-4726`.
- Chat refinement planning/streaming costs at `includes/class-gutenberg-sidebar.php:5360-5368` and `5516-5524`.
- Meta description cost at `includes/class-gutenberg-sidebar.php:6316-6324`.
- Intent detection cost at `includes/class-gutenberg-sidebar.php:6708-6716`.
Impact:
- The cost ledger can still show `unknown` provider for several real AI workflows.
- Provider/fallback transparency can differ between UI responses and persisted cost rows.
- Future fixes remain fragile while some paths use `track_ai_cost()` and others manually fire a shorter hook contract.
Recommended fix:
- Convert all internal AI cost tracking in `class-gutenberg-sidebar.php` to `track_ai_cost()` where the provider result is in scope.
- Where provider result is not currently in scope, pass it through with the generated response, or explicitly document why attribution is unavailable.
- Add a static guard that fails when `do_action( 'wp_aw_after_api_request' )` is called without the full provider/session/status contract outside the central helper.
### P2: Provider Metadata Coverage Still Has Stream Completion Gaps
The frontend now applies provider metadata in the branches called out by the twelfth pass:
- `assets/js/sidebar.js:3698`
- `assets/js/sidebar.js:3932`
- `assets/js/sidebar.js:4821`
Remaining gaps:
- `assets/js/sidebar.js:4222-4224` handles another stream `complete` event and updates cost without calling `applyProviderMetadata(data)`.
- Some backend stream completion payloads still include only completion/cost fields and no provider metadata, including `includes/class-gutenberg-sidebar.php:3730-3732`, `4824-4827`, and `5552-5555`.
Impact:
- Even when the frontend calls `applyProviderMetadata()`, some completion payloads do not contain metadata to apply.
- Provider badges can remain stale after specific streaming generation/refinement flows.
Recommended fix:
- Include `provider_metadata` on every stream `complete` payload that follows a provider call.
- Call `applyProviderMetadata(data)` in every frontend `data.type === 'complete'` branch that can receive provider-backed output.
- For completion events that are intentionally non-provider events, add a short comment so future audits do not treat them as missing.
### P2: Live Browser Verification Is Still Required
Static checks passed, but no live WordPress editor workflow evidence was found.
The next browser pass should verify:
- Legacy `_wpaw_chat_history` migrates through `/conversation/{post_id}` without fatal error.
- Sidebar conversation state persists after editor reload.
- Provider badge updates after chat, clarity, planning, generation, block refinement, chat refinement, meta, keyword, intent, and improvement actions.
- Cost log rows include provider/session/status for the same actions.
- Model settings changes affect generated requests.
- Unauthorized REST access remains denied.
### P3: Model Presets Are Documented As Curated, But Still Duplicated
The twelfth criteria allowed model presets/default ownership to be centralized or documented as intentional. The implementation chose documentation:
- `assets/js/settings-v2.js:32-34` says presets are curated product decisions, not registry-derived.
- `includes/class-settings.php:1026` says the legacy inline presets are curated, not registry-derived.
This is acceptable as a product decision, but it leaves duplicated preset data in active code:
- Active Settings V2 presets are hard-coded at `assets/js/settings-v2.js:35-59`.
- Legacy settings presets are hard-coded at `includes/class-settings.php:1027-1051`.
- The legacy settings class can still be instantiated at `wp-agentic-writer.php:100-104`.
Impact:
- Presets can drift between Settings V2 and legacy settings.
- Model updates still require edits in more than one place.
Recommended fix:
- If curated presets remain intentional, centralize them in one PHP source and localize them into both UIs.
- If legacy settings is only a fallback, add a deprecation note and a smaller test surface for preset parity.
## Priority Queue
1. P0: Replace direct `new WP_Agentic_Writer_Context_Service()` with `WP_Agentic_Writer_Context_Service::get_instance()` in the canonical conversation migration path.
2. P1: Convert remaining seven-argument cost hooks to the full provider/session/status contract or `track_ai_cost()`.
3. P2: Finish provider metadata coverage for every stream `complete` branch and payload.
4. P2: Run the live WordPress editor browser verification pass.
5. P3: Centralize curated model presets or document legacy preset parity ownership.
## Completion Criteria For Next Pass
The next retrace can mark this pass complete when:
- Legacy chat migration through `/conversation/{post_id}` cannot fatal on the context service constructor.
- Static scan finds no short-form `wp_aw_after_api_request` calls in provider-backed workflows.
- Provider-backed stream completion payloads include metadata, and frontend completion branches apply it or explicitly de-scope it.
- Browser verification evidence exists for chat persistence, migration, provider badge updates, cost log attribution, model settings, and auth denial.

197
docs/architecture/PLUGIN_AUDIT_RETRACE_TWELFTH_PASS_2026-05-26.md Normal file
View File

@@ -0,0 +1,197 @@
# WP Agentic Writer Twelfth Retrace Audit
Audit date: 2026-05-26
Baseline retraced: `docs/architecture/PLUGIN_AUDIT_RETRACE_ELEVENTH_PASS_2026-05-26.md`
Scope: twelfth pass after eleventh-retrace implementation, covering chat/context continuity, provider transparency, model registry adoption, cost attribution, UI/UX readiness, and release verification.
Status: COMPLETE / RETRACED
Completion marker: 2026-05-26
Follow-up retrace: `docs/architecture/PLUGIN_AUDIT_RETRACE_THIRTEENTH_PASS_2026-05-26.md`
> This twelfth-pass report has been implemented and retraced. Keep this document as historical evidence only; use the thirteenth-pass report for current remaining work.
## Executive Summary
The eleventh-pass implementation closed several meaningful items:
- The sidebar now uses the canonical `/conversation/{post_id}` endpoint instead of the deprecated `/chat-history/{post_id}` endpoint.
- `applyProviderMetadata()` is now called from many previously missed frontend paths, including meta generation, summarization, intent detection, reformat blocks, and refine-from-chat completion.
- Model registry adoption is broader: Settings V2 fallback labels/defaults, OpenRouter constructor defaults, and image manager defaults now draw from `WPAW_Model_Registry`.
- PHP and JavaScript syntax checks pass.
No new P0 blocker was found.
The remaining risk is narrower, but one new regression-class issue appeared during retrace: moving the sidebar to `/conversation` bypasses the legacy chat migration path that still lives under the deprecated chat-history compatibility method. That can make old post-meta chat history disappear from the editor UI for legacy posts that have not yet been migrated into conversation sessions.
## Verification Performed
- PHP syntax check across plugin PHP files: passed.
- `node -c assets/js/sidebar.js`: passed.
- `node -c assets/js/settings-v2.js`: passed.
- `node -c assets/js/sidebar-utils.js`: passed.
- Static retrace of eleventh-pass findings against current code.
- Static sweep of provider metadata coverage, model registry usage, chat/context hydration, cost hook contracts, and stale compatibility code.
- No live WordPress editor/browser workflow was run in this pass.
## Eleventh-Pass Status Trace
| Eleventh-pass item | Current status | Evidence |
|---|---:|---|
| Sidebar still used deprecated `/chat-history` | Fixed in active sidebar | `assets/js/sidebar.js:673` now fetches `/conversation/${postId}`. |
| Provider metadata missing in meta/summarize/intent/reformat/refine paths | Mostly fixed | `applyProviderMetadata()` is now called at `assets/js/sidebar.js:596`, `1603`, `1647`, `2195`, and `2836`. |
| Model registry not adopted in Settings V2 fallbacks/OpenRouter/image defaults | Improved | Settings V2 and provider/image defaults now use registry-backed defaults; residual hard-coded presets remain. |
| Raw cost hook drift | Still open | Direct `do_action( 'wp_aw_after_api_request', ... )` calls remain, including seven-argument calls that lose provider attribution. |
| Browser verification | Still open | Syntax checks passed, but live editor workflows were not verified. |
## Remaining Findings
### P1: Canonical `/conversation` Endpoint Bypasses Legacy Chat Migration
The active sidebar moved to the canonical endpoint:
- `assets/js/sidebar.js:673` fetches `/conversation/${postId}`.
However, the canonical backend handler only reads an existing conversation session:
- `includes/class-gutenberg-sidebar.php:1408-1418` calls `get_session_by_post_id()` and returns an empty message list when no session exists.
- `includes/class-conversation-manager.php:200-218` returns `null` when no active session is found.
The legacy migration behavior still exists, but only in the deprecated compatibility path:
- `includes/class-gutenberg-sidebar.php:1479-1492` reads `_wpaw_chat_history`, calls `migrate_legacy_chat_history()`, and returns migrated messages.
Impact:
- Legacy posts that still have `_wpaw_chat_history` but no conversation session can now hydrate as empty in the sidebar.
- This can look like conversation loss after the eleventh-pass fix, even though the data still exists in post meta.
- The deprecated endpoint has the safer migrate-on-read behavior, while the canonical endpoint does not.
Recommended fix:
- Make `handle_get_conversation_by_post()` use the same migrate-on-read behavior when no session exists and `_wpaw_chat_history` is present.
- Return the migrated `session_id`, `post_id`, `has_session: true`, and messages after migration.
- Add a regression test or fixture for a post with only `_wpaw_chat_history` and no `wpaw_conversations` row.
### P1: Cost Ledger Still Loses Provider Attribution For Some AI Actions
The centralized `track_ai_cost()` helper exists, but some AI actions still use the raw hook with the old seven-argument shape.
Examples:
- `includes/class-keyword-suggester.php:32-34` obtains a provider result, but `includes/class-keyword-suggester.php:121-129` fires `wp_aw_after_api_request` without provider, session, or status arguments.
- `includes/class-gutenberg-sidebar.php:6862-6864` obtains the provider for improvement analysis, but `includes/class-gutenberg-sidebar.php:6871-6879` also fires a seven-argument cost hook.
- `includes/class-cost-tracker.php:50` registers the listener with nine accepted arguments, so omitted provider fields fall back to `unknown`.
Impact:
- Cost rows for keyword suggestions and improvement suggestions can under-report provider and fallback status.
- Provider/cost dashboards can show `unknown` for actions where provider metadata was actually available.
- This weakens the new provider transparency work because the UI response and ledger do not always agree.
Recommended fix:
- Replace these raw hook calls with `track_ai_cost()` or a shared public cost helper.
- Where the helper is not accessible, pass the full nine-argument hook contract, including provider result, session id when available, and status.
- Add a static check for seven-argument `wp_aw_after_api_request` calls.
### P2: Provider Metadata UI Coverage Is Much Better, But Not Exhaustive
The eleventh-pass implementation fixed the major previously listed misses:
- Meta generation calls `applyProviderMetadata(data)` at `assets/js/sidebar.js:596`.
- Summarization calls it at `assets/js/sidebar.js:1603`.
- Intent detection calls it at `assets/js/sidebar.js:1647`.
- Reformat blocks calls it at `assets/js/sidebar.js:2195`.
- Refine-from-chat completion calls it at `assets/js/sidebar.js:2836`.
Remaining missed or duplicate paths:
- One stream completion branch at `assets/js/sidebar.js:3697-3710` emits a completion payload with `totalCost` only, so the frontend has no provider metadata to apply.
- Another completion handler at `assets/js/sidebar.js:3930-3945` updates cost/timeline without applying provider metadata.
- The clarity check path at `assets/js/sidebar.js:4818-4825` parses `clarityData` but does not apply provider metadata from the response.
Impact:
- The provider badge is now reliable for many common flows, but can still remain stale after some generation or clarity workflows.
- Users may see correct cost movement while the provider/fallback display still reflects a previous request.
Recommended fix:
- Ensure every backend AI response and stream completion payload includes `provider_metadata` when a provider was involved.
- Call `applyProviderMetadata(data)` immediately after every AI JSON response parse and every stream `complete` event that can carry metadata.
- For endpoints intentionally without provider metadata, document that in code next to the parse/complete branch.
### P2: Model Registry Still Has Residual Hard-Coded Defaults And Presets
Registry adoption is now strong in the active PHP defaults, but not complete:
- `assets/js/settings-v2.js:32-58` still defines budget/balanced/premium presets with hard-coded model IDs.
- `includes/class-settings.php:72-78`, `98-117`, `138-140`, `197`, and `1029-1049` still contain legacy settings defaults and presets with hard-coded model IDs.
- `wp-agentic-writer.php:100-104` can still instantiate the legacy settings class if Settings V2 is unavailable.
Some hard-coded model strings are acceptable when they are display labels, compatibility checks, pricing keys, or provider-specific suggestions. The remaining concern is default/preset ownership.
Impact:
- The registry is not yet the only source of model defaults users can activate.
- JS presets and legacy settings can drift from the PHP registry.
- Future model migrations still require edits in more than one place.
Recommended fix:
- Localize presets from `WPAW_Model_Registry::get_frontend_data()` or add explicit registry preset support.
- Either retire the legacy `WP_Agentic_Writer_Settings` path or make it read registry defaults.
- Keep provider suggestion maps and pricing tables separate, but name them as suggestions/pricing rather than defaults.
### P2: Live Browser Verification Is Still The Final Release Gate
Static checks passed, but the plugin still needs a live editor pass before calling the audit chain fully complete.
Manual verification should cover:
- Sidebar opens in the block editor and survives page reload.
- Existing legacy post-meta chat history migrates and appears through `/conversation/{post_id}`.
- New chat messages persist, reload, and retain session id continuity.
- Provider/fallback badges update after chat, clarity, planning, generation, refinement, summarize, reformat, meta, keyword, and improvement actions.
- Cost totals update in the UI and cost ledger with provider/session/status fields populated.
- Model setting changes affect generated requests and visible provider metadata.
- Unauthorized REST access remains denied.
### P3: Stale Compatibility Comments And Backup Files Can Create Audit Noise
The deprecated chat-history docblock is now stale:
- `includes/class-gutenberg-sidebar.php:1349-1350` says the endpoint does not use the conversations table, but the implementation delegates through conversation/session migration behavior.
Also, backup JavaScript files still reference old endpoint behavior:
- `assets/js/sidebar.js.backup`
- `assets/js/sidebar.js.bak`
Impact:
- Future retraces can mistake backup files or stale comments for active behavior.
- If backup files are accidentally packaged, old endpoint usage can leak into distribution artifacts.
Recommended fix:
- Update the deprecated endpoint docblock to describe the actual compatibility behavior.
- Exclude backup files from packaging or remove them after confirming they are not needed.
## Priority Queue
1. P1: Add migrate-on-read behavior to `/conversation/{post_id}` for legacy `_wpaw_chat_history`.
2. P1: Fix seven-argument cost hooks for keyword and improvement suggestions so provider attribution is preserved.
3. P2: Finish provider metadata propagation for remaining stream completion and clarity branches.
4. P2: Move active JS presets and legacy settings defaults under the model registry, or explicitly retire/de-scope legacy settings.
5. P2: Run live WordPress editor browser verification.
6. P3: Clean stale chat-history comments and backup endpoint references.
## Completion Criteria For Next Pass
The next retrace can mark this pass complete when:
- `/conversation/{post_id}` migrates legacy chat history when no session exists.
- Keyword and improvement suggestion cost rows include provider, fallback, session/status where available.
- Provider metadata is applied or explicitly de-scoped for every AI response path.
- Remaining model preset/default ownership is either centralized in the registry or documented as intentional.
- A live editor verification note exists with the exact workflows checked.