31 KiB
31 KiB
WP Agentic Writer - Comprehensive Agentic Audit Report
Audit Date: January 21, 2026
Auditor: Cascade AI
Plugin Version: 0.1.0
Goal: Evaluate "Agentic-like IDE" capabilities in WordPress Gutenberg editor
Executive Summary
WP Agentic Writer is a sophisticated AI-powered writing assistant with a solid foundation for agentic workflows. The plugin successfully implements the core "plan-first" approach: Scribble → Research → Plan → Execute → Refine. However, several gaps exist between the current implementation and a truly IDE-like agentic experience.
Overall Agentic Score: 7.5/10
Strengths
- ✅ Multi-phase workflow (planning → execution)
- ✅ Clarification quiz for context gathering
- ✅
@mentionsystem for block targeting (IDE-like) - ✅ Slash commands (
/add below,/add above,/append code) - ✅ Real-time streaming with timeline progress
- ✅ Section-aware refinement with context
- ✅ Plan preview with diff-style actions
- ✅ Cost tracking and budget management
Gaps Identified
- ⚠️ No undo/redo for AI changes
- ⚠️ No diff view before applying changes
- ⚠️ No "Accept/Reject" workflow for refinements
- ⚠️ Limited autonomous decision-making
- ⚠️ No agent memory across sessions (post-level only)
- ⚠️ No multi-step autonomous execution
- ⚠️ Missing keyboard shortcuts for power users
Part 1: Current Architecture Analysis
1.1 Core Workflow Trace
┌─────────────────────────────────────────────────────────────────┐
│ USER INPUT (Chat Sidebar) │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ CLARITY CHECK (if enabled) │
│ - Evaluates 7 context categories │
│ - Generates clarification quiz if confidence < threshold │
│ - Categories: outcome, audience, tone, depth, expertise, │
│ content type, POV │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ MODE DETECTION │
│ - "writing" mode → Full article generation │
│ - "planning" mode → Outline only │
│ - Refinement detected → Chat refinement flow │
│ - Insert command detected → Add block flow │
└─────────────────────────────────────────────────────────────────┘
│
┌───────────────┴───────────────┐
▼ ▼
┌──────────────────────┐ ┌──────────────────────┐
│ PLAN GENERATION │ │ CHAT REFINEMENT │
│ - Stream outline │ │ - @mention resolve │
│ - Section breakdown │ │ - Edit plan create │
│ - Auto-execute │ │ - Block replacement │
└──────────────────────┘ └──────────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────────┐
│ GUTENBERG BLOCK OPERATIONS │
│ - createBlock(), insertBlocks(), replaceBlocks() │
│ - Section tracking (sectionBlocksRef) │
│ - Real-time editor updates │
└─────────────────────────────────────────────────────────────────┘
1.2 File Structure Analysis
| File | Purpose | Lines | Agentic Features |
|---|---|---|---|
sidebar.js |
Main UI + logic | 4,359 | @mentions, slash commands, plan execution |
class-gutenberg-sidebar.php |
REST API + AI calls | 4,274 | Streaming, refinement, memory |
class-openrouter-provider.php |
AI provider | 566 | Multi-model, web search |
block-refine.js |
Toolbar integration | 115 | @chat button on blocks |
sidebar.css |
Styling | 1,817 | Timeline, quiz UI |
1.3 Agent Modes
| Mode | Description | Agentic Level |
|---|---|---|
| Writing | Full article generation from prompt | ⭐⭐⭐ Medium |
| Planning | Outline-only with manual execution | ⭐⭐ Low |
| Refinement | Block-level changes via @mentions | ⭐⭐⭐⭐ High |
Part 2: UI/UX Analysis
2.1 Sidebar Interface
Current Structure:
┌─────────────────────────────────────────┐
│ 💬 Chat │ ⚙️ Config │ 💰 Cost │ ← Tab Navigation
├─────────────────────────────────────────┤
│ [Status Bar - Mode + Cost] │
├─────────────────────────────────────────┤
│ ┌─────────────────────────────────────┐ │
│ │ Messages Area (scrollable) │ │
│ │ - User messages │ │
│ │ - Assistant responses │ │
│ │ - Timeline entries (progress) │ │
│ │ - Plan cards │ │
│ │ - Error messages │ │
│ └─────────────────────────────────────┘ │
├─────────────────────────────────────────┤
│ ┌─────────────────────────────────────┐ │
│ │ Input Area │ │
│ │ - Mode selector │ │
│ │ - Textarea with @mention support │ │
│ │ - Send button │ │
│ └─────────────────────────────────────┘ │
└─────────────────────────────────────────┘
✅ Good:
- Dark theme matches IDE aesthetic
- Monospace fonts for code-like feel
- Timeline progress shows "agent thinking"
- @mention autocomplete is discoverable
❌ Issues:
- No keyboard shortcuts - Power users expect Cmd+Enter to send
- No command palette - IDEs have Cmd+Shift+P for quick actions
- Config tab is underutilized - Only article length selector
- No block outline view - Can't see article structure at a glance
2.2 Block Toolbar Integration
Current: @chat button in block toolbar sends mention to sidebar
Issue: The button label is just "@chat" - unclear what it does
Recommendation: Rename to "Refine with AI" or add tooltip
2.3 Clarification Quiz UX
Current Flow:
- Quiz appears as modal overlay in chat
- Radio buttons for predefined options
- Progress bar shows completion
- Skip button available
✅ Good:
- Predefined options reduce friction
- Progress indicator is clear
- Fallback questions when AI fails
❌ Issues:
- No "Don't ask again" option - Users may want to skip permanently
- Quiz interrupts flow - Could be inline instead of modal
- No learning from previous posts - Always starts fresh
Part 3: Functionality Deep Dive
3.1 Generation Flow
Strengths:
- Streaming responses with real-time timeline
- Section-by-section execution
- Automatic title update from AI
- Resume capability after errors
Gaps:
- No preview before insertion - Blocks appear directly
- No staged commits - Can't review all changes before applying
- No branch/version history - Can't revert to previous state
3.2 Refinement System
Supported Commands:
| Command | Action |
|---|---|
@this |
Current selected block |
@previous |
Block before current |
@next |
Block after current |
@all |
All content blocks |
@paragraph-N |
Nth paragraph |
@heading-N |
Nth heading |
@list-N |
Nth list |
/add below @block |
Insert paragraph below |
/add above @block |
Insert paragraph above |
/append code @block |
Insert code block |
/reformat @block |
Convert markdown to blocks |
✅ This is very IDE-like!
Gaps:
- No
@code-N- Can't target code blocks directly - No
@image-N- Can't target images - No range selection - Can't say
@paragraph-1:3for range - No diff preview - Changes apply immediately
3.3 Edit Plan System
Current:
// Edit plan structure
{
"summary": "short summary",
"actions": [
{"action": "keep", "blockId": "..."},
{"action": "replace", "blockId": "...", "blockType": "...", "content": "..."},
{"action": "insert_after", "blockId": "...", "blockType": "...", "content": "..."},
{"action": "delete", "blockId": "..."}
]
}
✅ Good:
- Diff-style action preview
- Click-to-scroll to target block
- Execute/Cancel buttons
❌ Issues:
- All-or-nothing execution - Can't apply individual actions
- No partial accept - Can't accept some, reject others
- No inline editing - Can't modify plan before applying
3.4 Memory System
Current:
- Post-level memory stored in
_wpaw_memorymeta - Contains: summary, last_prompt, last_intent
- Chat history persisted per post
Gaps:
- No global memory - Can't learn user preferences across posts
- No style guide storage - Can't save writing style preferences
- No context from other posts - Can't reference previous work
Part 4: Agentic Gaps & Recommendations
4.1 Critical Missing Features (High Priority)
4.1.1 Undo/Redo for AI Changes
Problem: No way to revert AI changes without manual Cmd+Z
Solution:
// Add undo stack for AI operations
const aiUndoStack = [];
const executeWithUndo = (operation) => {
const snapshot = captureEditorState();
aiUndoStack.push(snapshot);
operation();
};
// UI: Add "Undo AI" button in timeline entries
4.1.2 Diff View Before Apply
Problem: Users can't see what will change before it happens
Solution:
- Add "Preview Changes" mode
- Show side-by-side or inline diff
- GitHub-style green/red highlighting
4.1.3 Accept/Reject Workflow
Problem: Changes apply immediately with no approval
Solution:
┌─────────────────────────────────────────┐
│ AI suggests: Replace paragraph 3 │
│ │
│ Before: "The old content..." │
│ After: "The new content..." │
│ │
│ [Accept] [Reject] [Edit] [Skip] │
└─────────────────────────────────────────┘
4.2 Agentic Enhancements (Medium Priority)
4.2.1 Autonomous Multi-Step Execution
Current: User must approve each step
Agentic: Agent completes entire task autonomously
Recommendation: Add "Full Auto" mode
const agentModes = {
'supervised': 'Approve each change', // Current
'semi-auto': 'Approve plan, auto-execute', // New
'full-auto': 'Complete task autonomously' // New (advanced)
};
4.2.2 Agent Memory & Learning
Current: No learning across sessions
Agentic: Remember user preferences, writing style
Recommendation:
// Global user preferences in wp_usermeta
update_user_meta($user_id, '_wpaw_preferences', [
'tone' => 'professional',
'avoid_words' => ['leverage', 'synergy'],
'preferred_length' => 'medium',
'always_include' => ['code examples'],
]);
4.2.3 Context-Aware Suggestions
Current: Agent only responds to commands
Agentic: Agent proactively suggests improvements
Recommendation:
- Analyze article on idle
- Suggest improvements in sidebar
- "I noticed paragraph 3 could be clearer. Want me to refine it?"
4.3 IDE-Like Features (Medium Priority)
4.3.1 Keyboard Shortcuts
| Shortcut | Action |
|---|---|
Cmd+Enter |
Send message |
Cmd+Shift+P |
Command palette |
Cmd+/ |
Quick refine selected block |
Cmd+G |
Generate from selection |
Escape |
Cancel current operation |
4.3.2 Command Palette
┌─────────────────────────────────────────┐
│ > _ │
├─────────────────────────────────────────┤
│ 📝 Generate article from prompt │
│ ✏️ Refine selected block │
│ 📋 Create outline only │
│ 🔄 Regenerate current section │
│ 🌐 Enable web search │
│ ⚙️ Open settings │
└─────────────────────────────────────────┘
4.3.3 Block Outline Panel
┌─────────────────────────────────────────┐
│ ARTICLE STRUCTURE │
├─────────────────────────────────────────┤
│ ▼ Introduction │
│ ├─ paragraph-1 │
│ └─ paragraph-2 │
│ ▼ Getting Started │
│ ├─ heading-2 │
│ ├─ paragraph-3 │
│ └─ code-1 │
│ ▼ Advanced Usage │
│ ├─ heading-3 │
│ └─ list-1 │
└─────────────────────────────────────────┘
4.4 UX Improvements (Lower Priority)
4.4.1 Inline Refinement
Current: Must use sidebar for all refinements
Improvement: Click block → inline popover with quick actions
4.4.2 Streaming Preview
Current: Content appears in editor directly
Improvement: Show in preview pane first, then "Apply All"
4.4.3 Smart Suggestions Bar
Show contextual actions based on selection:
┌─────────────────────────────────────────┐
│ 💡 Make concise │ 🔄 Rephrase │ 📝 Expand │
└─────────────────────────────────────────┘
Part 5: Technical Debt & Code Quality
5.1 Identified Issues
sidebar.jsis 4,359 lines - Should be split into modules- Mixed concerns - UI, state, API calls in same file
- No TypeScript - Type safety would prevent bugs
- Hardcoded strings - Should use i18n throughout
5.2 Recommendations
-
Modularize sidebar.js:
hooks/useChat.jshooks/usePlan.jshooks/useRefinement.jscomponents/ChatTab.jscomponents/ConfigTab.jsutils/blockHelpers.js
-
Add error boundaries - React error boundaries for graceful failures
-
Implement proper state management - Consider Redux or Zustand
Part 6: Prioritized Action Items
Tier 1: Critical (Do First)
| Item | Effort | Impact | Status |
|---|---|---|---|
| Add Undo for AI changes | Medium | High | ❌ Not Implemented |
| Add Accept/Reject workflow | Medium | High | ✅ Implemented (Apply/Cancel buttons) |
| Add Cmd+Enter to send | Low | Medium | ✅ Implemented (line 2326 sidebar.js) |
| Add diff preview for edit plans | Medium | High | ✅ Implemented (edit_plan type with before/after) |
Note: Cross-verified on Jan 21, 2026. Only Undo for AI changes remains to be implemented in Tier 1.
Tier 2: Important (Do Next)
| Item | Effort | Impact |
|---|---|---|
| Command palette (Cmd+Shift+P) | Medium | High |
| Per-action accept/reject in plans | Medium | Medium |
| Block outline panel | Medium | Medium |
| Global user preferences | Low | Medium |
Tier 3: Nice to Have
| Item | Effort | Impact |
|---|---|---|
| Inline refinement popover | High | Medium |
| Streaming preview pane | High | Medium |
| Smart suggestions bar | Medium | Low |
| Full-auto mode | High | Low |
Part 7: Conclusion
What's Already Agentic ✅
- @mention system - Best-in-class block targeting
- Slash commands - IDE-like quick actions
- Clarification quiz - Proactive context gathering
- Edit plan preview - Shows intent before action
- Section tracking - Maintains document structure
What's Missing for True Agentic ❌
- Approval workflow - Changes should be reviewable
- Undo/history - Need to revert AI mistakes
- Autonomous execution - Agent should complete tasks independently
- Learning/memory - Should improve over time
- Keyboard-first UX - Power users need shortcuts
Final Recommendation
The plugin has strong agentic bones but needs the safety net features that make IDEs trustworthy:
- Immediate wins: Keyboard shortcuts + Undo button
- Medium-term: Accept/Reject workflow + Command palette
- Long-term: Autonomous mode + Learning system
The goal should be: "I can trust this agent to make changes because I can always review, approve, or revert."
Part 8: Proactive AI Suggestions (NEW REQUIREMENT)
8.1 Current State
Problem: Agent only responds to commands - purely reactive.
8.2 Target State
Goal: Agent proactively analyzes content and suggests improvements.
8.3 Implementation Specification
8.3.1 Idle Analysis Trigger
// Trigger analysis after user stops editing for N seconds
const IDLE_THRESHOLD_MS = 5000; // 5 seconds
let idleTimer = null;
const startIdleAnalysis = () => {
clearTimeout(idleTimer);
idleTimer = setTimeout(() => {
analyzeArticleForSuggestions();
}, IDLE_THRESHOLD_MS);
};
// Hook into editor changes
wp.data.subscribe(() => {
startIdleAnalysis();
});
8.3.2 Suggestion Types
| Category | Example Suggestion |
|---|---|
| Clarity | "Paragraph 3 could be clearer. Want me to simplify it?" |
| Flow | "The transition between sections 2 and 3 feels abrupt." |
| Depth | "This section could use more examples or data." |
| SEO | "Consider adding keyword 'X' to heading 2." |
| Structure | "This article could benefit from a summary section." |
| Engagement | "Consider adding a question to engage readers here." |
8.3.3 UI Component
┌─────────────────────────────────────────────────┐
│ 💡 Suggestion │
│ │
│ "I noticed paragraph 3 could be clearer. │
│ Want me to refine it?" │
│ │
│ [Apply] [Dismiss] [Don't show again] │
└─────────────────────────────────────────────────┘
8.3.4 Backend Endpoint
// New REST endpoint: /analyze-for-suggestions
public function analyze_for_suggestions() {
$blocks = $this->get_all_blocks();
$prompt = "Analyze this article and suggest 1-3 improvements...";
// Return structured suggestions
}
8.3.5 User Preferences
- Toggle: "Enable proactive suggestions"
- Frequency: "Aggressive / Balanced / Minimal"
- Categories: Checkboxes for which suggestion types to show
Part 9: SEO Specialist Capabilities (NEW REQUIREMENT)
9.1 Vision
Agentic Writer should not only write well but write SEO-optimized content that ranks.
9.2 SEO Feature Matrix
| Feature | Description | Priority |
|---|---|---|
| Keyword Analysis | Analyze target keyword, suggest density | High |
| Keyword Placement | Ensure keyword in title, H1, first paragraph | High |
| Heading Structure | Validate H1→H2→H3 hierarchy | High |
| Meta Generation | Auto-generate meta title & description | High |
| Readability Score | Flesch-Kincaid or similar | Medium |
| Internal Linking | Suggest links to other posts | Medium |
| Image Alt Text | Auto-generate SEO-friendly alt text | Medium |
| Schema Markup | Suggest FAQ, HowTo, Article schema | Medium |
| Competitor Analysis | Compare with top-ranking articles | Low |
| SERP Preview | Show how it will appear in Google | Low |
9.3 SEO Workflow Integration
9.3.1 Pre-Writing Phase
┌─────────────────────────────────────────────────┐
│ 🎯 SEO Setup │
│ │
│ Target Keyword: [_______________] │
│ Secondary Keywords: [_______________] │
│ │
│ ☑ Analyze competition before writing │
│ ☑ Include keyword in title │
│ ☑ Suggest internal links │
│ │
│ [Analyze Competition] [Skip to Writing] │
└─────────────────────────────────────────────────┘
9.3.2 During Writing (Real-time)
- Keyword density indicator in sidebar
- Heading structure validator
- Reading time & word count
- Readability score (live)
9.3.3 Post-Writing (SEO Audit)
┌─────────────────────────────────────────────────┐
│ 📊 SEO Score: 78/100 │
│ │
│ ✅ Keyword in title │
│ ✅ Keyword in first paragraph │
│ ⚠️ Keyword density low (0.8%, target 1-2%) │
│ ❌ Missing meta description │
│ ❌ No internal links found │
│ ✅ Proper heading hierarchy │
│ ⚠️ Images missing alt text (2 of 3) │
│ │
│ [Fix All Issues] [Generate Meta] [Add Links] │
└─────────────────────────────────────────────────┘
9.4 SEO-Aware System Prompts
Modify the plan generation prompt to include SEO considerations:
You are an SEO-optimized content writer. When creating content:
1. KEYWORD PLACEMENT:
- Include target keyword in H1 and first 100 words
- Use keyword naturally 1-2% density
- Include semantic variations
2. STRUCTURE:
- Use proper heading hierarchy (H1→H2→H3)
- Include table of contents for long articles
- Use bullet points and numbered lists
- Keep paragraphs under 150 words
3. ENGAGEMENT:
- Start with a hook
- Use questions to engage readers
- Include actionable takeaways
4. TECHNICAL:
- Suggest descriptive image alt text
- Recommend internal link opportunities
- Optimize for featured snippets where applicable
9.5 Post Config Additions
const seoConfig = {
target_keyword: '',
secondary_keywords: [],
enable_seo_mode: true,
keyword_density_target: 1.5, // percentage
min_word_count: 1500,
include_meta: true,
include_schema: false,
internal_linking: true,
};
9.6 New REST Endpoints
| Endpoint | Purpose |
|---|---|
/seo/analyze-keyword |
Get keyword difficulty, volume |
/seo/audit-content |
Full SEO audit of current content |
/seo/generate-meta |
Generate meta title/description |
/seo/suggest-links |
Find internal linking opportunities |
/seo/competitor-analysis |
Analyze top-ranking content |
Part 10: AI Model Recommendations (NEW REQUIREMENT)
10.1 Purpose
Help users choose the cheapest AND best models for agentic workflows.
10.2 Model Tiers by Use Case
10.2.1 Planning Phase (Fast, Cheap)
| Model | Cost (per 1M tokens) | Speed | Quality | Recommendation |
|---|---|---|---|---|
google/gemini-2.0-flash-exp |
~$0.10 in / $0.40 out | ⚡ Very Fast | Good | Best Value |
google/gemini-flash-1.5 |
~$0.075 in / $0.30 out | ⚡ Very Fast | Good | Budget Option |
anthropic/claude-3-haiku |
~$0.25 in / $1.25 out | Fast | Good | Reliable |
openai/gpt-4o-mini |
~$0.15 in / $0.60 out | Fast | Good | Alternative |
10.2.2 Execution Phase (Quality Critical)
| Model | Cost (per 1M tokens) | Speed | Quality | Recommendation |
|---|---|---|---|---|
anthropic/claude-sonnet-4 |
~$3 in / $15 out | Medium | Excellent | Best for Writing |
anthropic/claude-3.5-sonnet |
~$3 in / $15 out | Medium | Excellent | Proven Quality |
openai/gpt-4o |
~$2.50 in / $10 out | Medium | Excellent | Alternative |
google/gemini-pro-1.5 |
~$1.25 in / $5 out | Medium | Very Good | Cost-Conscious |
10.2.3 Image Generation
| Model | Cost (per image) | Speed | Quality | Recommendation |
|---|---|---|---|---|
black-forest-labs/flux-schnell |
~$0.003 | ⚡ Very Fast | Good | Best Value |
black-forest-labs/flux-1.1-pro |
~$0.04 | Fast | Excellent | Premium |
openai/dall-e-3 |
~$0.04-0.08 | Medium | Excellent | Alternative |
10.3 Recommended Configurations
10.3.1 Budget-Conscious Setup (~$0.05 per article)
Planning Model: google/gemini-2.0-flash-exp
Execution Model: google/gemini-pro-1.5
Image Model: black-forest-labs/flux-schnell
10.3.2 Balanced Setup (~$0.15 per article)
Planning Model: google/gemini-2.0-flash-exp
Execution Model: anthropic/claude-3.5-sonnet
Image Model: black-forest-labs/flux-schnell
10.3.3 Premium Quality Setup (~$0.30+ per article)
Planning Model: anthropic/claude-3-haiku
Execution Model: anthropic/claude-sonnet-4
Image Model: black-forest-labs/flux-1.1-pro
10.4 Model Selection UI
┌─────────────────────────────────────────────────┐
│ 🤖 Model Configuration │
│ │
│ Preset: [Budget ▾] [Balanced ▾] [Premium ▾] │
│ │
│ Planning Model: │
│ [google/gemini-2.0-flash-exp ▾] │
│ 💰 ~$0.10/1M tokens • ⚡ Fast │
│ │
│ Execution Model: │
│ [anthropic/claude-sonnet-4 ▾] │
│ 💰 ~$3/1M tokens • ✨ Best Quality │
│ │
│ Image Model: │
│ [black-forest-labs/flux-schnell ▾] │
│ 💰 ~$0.003/image • ⚡ Fast │
│ │
│ Estimated cost per article: ~$0.15 │
└─────────────────────────────────────────────────┘
10.5 Smart Model Suggestions
The plugin should analyze usage patterns and suggest:
-
If budget is tight:
"You've spent $45 this month. Switch to Budget preset to save ~60%."
-
If quality issues detected:
"Multiple refinements on recent articles. Consider upgrading execution model."
-
If speed is priority:
"For faster generation, switch planning to Gemini Flash."
Part 11: Updated Roadmap
Phase 1: Complete Tier 1 ✅ COMPLETED
- Cmd+Enter to send
- Accept/Reject workflow (Apply/Cancel)
- Diff preview for edit plans
- Undo for AI changes (sidebar.js: aiUndoStack, pushUndoSnapshot, undoLastAiOperation)
- Budget tracker enhanced (Cost tab with refresh, remaining display, warnings)
- Settings page revamp (Modern card-based UI with preset configurations)
Phase 2: SEO Foundation ✅ COMPLETED
- Add SEO config fields (focus keyword, secondary keywords, meta description)
- Integrate SEO considerations into system prompts (build_seo_context)
- Add keyword density indicator (in SEO audit)
- Add SEO audit endpoint (/seo-audit/{post_id})
- Add meta generation (/generate-meta with AI)
Phase 3: Proactive Suggestions (2-3 weeks)
- Implement idle detection
- Create suggestion analysis endpoint
- Add suggestion UI component
- Add user preferences for suggestions
Phase 4: Model Recommendations ✅ COMPLETED
- Add preset configurations (Budget, Balanced, Premium)
- Add model selection UI with cost estimates
- Add smart suggestions based on usage
Phase 5: Tier 2 Features (3-4 weeks)
- Command palette (Cmd+Shift+P)
- Per-action accept/reject in plans
- Block outline panel
- Global user preferences
Report Updated: January 22, 2026
Implementation Status: Phase 1, Phase 2 (SEO Foundation) & Phase 4 completed. Ready for Phase 3 (Proactive Suggestions).