diff --git a/AGENTIC_AUDIT_REPORT.md b/AGENTIC_AUDIT_REPORT.md new file mode 100644 index 0000000..94f8ecd --- /dev/null +++ b/AGENTIC_AUDIT_REPORT.md @@ -0,0 +1,791 @@ +# WP Agentic Writer - Comprehensive Agentic Audit Report + +**Audit Date:** January 21, 2026 +**Auditor:** Cascade AI +**Plugin Version:** 0.1.0 +**Goal:** Evaluate "Agentic-like IDE" capabilities in WordPress Gutenberg editor + +--- + +## Executive Summary + +WP Agentic Writer is a sophisticated AI-powered writing assistant with a **solid foundation** for agentic workflows. The plugin successfully implements the core "plan-first" approach: `Scribble → Research → Plan → Execute → Refine`. However, several gaps exist between the current implementation and a truly **IDE-like agentic experience**. + +**Overall Agentic Score: 7.5/10** + +### Strengths +- ✅ Multi-phase workflow (planning → execution) +- ✅ Clarification quiz for context gathering +- ✅ `@mention` system for block targeting (IDE-like) +- ✅ Slash commands (`/add below`, `/add above`, `/append code`) +- ✅ Real-time streaming with timeline progress +- ✅ Section-aware refinement with context +- ✅ Plan preview with diff-style actions +- ✅ Cost tracking and budget management + +### Gaps Identified +- ⚠️ No undo/redo for AI changes +- ⚠️ No diff view before applying changes +- ⚠️ No "Accept/Reject" workflow for refinements +- ⚠️ Limited autonomous decision-making +- ⚠️ No agent memory across sessions (post-level only) +- ⚠️ No multi-step autonomous execution +- ⚠️ Missing keyboard shortcuts for power users + +--- + +## Part 1: Current Architecture Analysis + +### 1.1 Core Workflow Trace + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ USER INPUT (Chat Sidebar) │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ CLARITY CHECK (if enabled) │ +│ - Evaluates 7 context categories │ +│ - Generates clarification quiz if confidence < threshold │ +│ - Categories: outcome, audience, tone, depth, expertise, │ +│ content type, POV │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ MODE DETECTION │ +│ - "writing" mode → Full article generation │ +│ - "planning" mode → Outline only │ +│ - Refinement detected → Chat refinement flow │ +│ - Insert command detected → Add block flow │ +└─────────────────────────────────────────────────────────────────┘ + │ + ┌───────────────┴───────────────┐ + ▼ ▼ +┌──────────────────────┐ ┌──────────────────────┐ +│ PLAN GENERATION │ │ CHAT REFINEMENT │ +│ - Stream outline │ │ - @mention resolve │ +│ - Section breakdown │ │ - Edit plan create │ +│ - Auto-execute │ │ - Block replacement │ +└──────────────────────┘ └──────────────────────┘ + │ │ + ▼ ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ GUTENBERG BLOCK OPERATIONS │ +│ - createBlock(), insertBlocks(), replaceBlocks() │ +│ - Section tracking (sectionBlocksRef) │ +│ - Real-time editor updates │ +└─────────────────────────────────────────────────────────────────┘ +``` + +### 1.2 File Structure Analysis + +| File | Purpose | Lines | Agentic Features | +|------|---------|-------|------------------| +| `sidebar.js` | Main UI + logic | 4,359 | @mentions, slash commands, plan execution | +| `class-gutenberg-sidebar.php` | REST API + AI calls | 4,274 | Streaming, refinement, memory | +| `class-openrouter-provider.php` | AI provider | 566 | Multi-model, web search | +| `block-refine.js` | Toolbar integration | 115 | @chat button on blocks | +| `sidebar.css` | Styling | 1,817 | Timeline, quiz UI | + +### 1.3 Agent Modes + +| Mode | Description | Agentic Level | +|------|-------------|---------------| +| **Writing** | Full article generation from prompt | ⭐⭐⭐ Medium | +| **Planning** | Outline-only with manual execution | ⭐⭐ Low | +| **Refinement** | Block-level changes via @mentions | ⭐⭐⭐⭐ High | + +--- + +## Part 2: UI/UX Analysis + +### 2.1 Sidebar Interface + +**Current Structure:** +``` +┌─────────────────────────────────────────┐ +│ 💬 Chat │ ⚙️ Config │ 💰 Cost │ ← Tab Navigation +├─────────────────────────────────────────┤ +│ [Status Bar - Mode + Cost] │ +├─────────────────────────────────────────┤ +│ ┌─────────────────────────────────────┐ │ +│ │ Messages Area (scrollable) │ │ +│ │ - User messages │ │ +│ │ - Assistant responses │ │ +│ │ - Timeline entries (progress) │ │ +│ │ - Plan cards │ │ +│ │ - Error messages │ │ +│ └─────────────────────────────────────┘ │ +├─────────────────────────────────────────┤ +│ ┌─────────────────────────────────────┐ │ +│ │ Input Area │ │ +│ │ - Mode selector │ │ +│ │ - Textarea with @mention support │ │ +│ │ - Send button │ │ +│ └─────────────────────────────────────┘ │ +└─────────────────────────────────────────┘ +``` + +**✅ Good:** +- Dark theme matches IDE aesthetic +- Monospace fonts for code-like feel +- Timeline progress shows "agent thinking" +- @mention autocomplete is discoverable + +**❌ Issues:** +1. **No keyboard shortcuts** - Power users expect Cmd+Enter to send +2. **No command palette** - IDEs have Cmd+Shift+P for quick actions +3. **Config tab is underutilized** - Only article length selector +4. **No block outline view** - Can't see article structure at a glance + +### 2.2 Block Toolbar Integration + +**Current:** `@chat` button in block toolbar sends mention to sidebar + +**Issue:** The button label is just "@chat" - unclear what it does + +**Recommendation:** Rename to "Refine with AI" or add tooltip + +### 2.3 Clarification Quiz UX + +**Current Flow:** +1. Quiz appears as modal overlay in chat +2. Radio buttons for predefined options +3. Progress bar shows completion +4. Skip button available + +**✅ Good:** +- Predefined options reduce friction +- Progress indicator is clear +- Fallback questions when AI fails + +**❌ Issues:** +1. **No "Don't ask again" option** - Users may want to skip permanently +2. **Quiz interrupts flow** - Could be inline instead of modal +3. **No learning from previous posts** - Always starts fresh + +--- + +## Part 3: Functionality Deep Dive + +### 3.1 Generation Flow + +**Strengths:** +- Streaming responses with real-time timeline +- Section-by-section execution +- Automatic title update from AI +- Resume capability after errors + +**Gaps:** +1. **No preview before insertion** - Blocks appear directly +2. **No staged commits** - Can't review all changes before applying +3. **No branch/version history** - Can't revert to previous state + +### 3.2 Refinement System + +**Supported Commands:** +| Command | Action | +|---------|--------| +| `@this` | Current selected block | +| `@previous` | Block before current | +| `@next` | Block after current | +| `@all` | All content blocks | +| `@paragraph-N` | Nth paragraph | +| `@heading-N` | Nth heading | +| `@list-N` | Nth list | +| `/add below @block` | Insert paragraph below | +| `/add above @block` | Insert paragraph above | +| `/append code @block` | Insert code block | +| `/reformat @block` | Convert markdown to blocks | + +**✅ This is very IDE-like!** + +**Gaps:** +1. **No `@code-N`** - Can't target code blocks directly +2. **No `@image-N`** - Can't target images +3. **No range selection** - Can't say `@paragraph-1:3` for range +4. **No diff preview** - Changes apply immediately + +### 3.3 Edit Plan System + +**Current:** +```javascript +// Edit plan structure +{ + "summary": "short summary", + "actions": [ + {"action": "keep", "blockId": "..."}, + {"action": "replace", "blockId": "...", "blockType": "...", "content": "..."}, + {"action": "insert_after", "blockId": "...", "blockType": "...", "content": "..."}, + {"action": "delete", "blockId": "..."} + ] +} +``` + +**✅ Good:** +- Diff-style action preview +- Click-to-scroll to target block +- Execute/Cancel buttons + +**❌ Issues:** +1. **All-or-nothing execution** - Can't apply individual actions +2. **No partial accept** - Can't accept some, reject others +3. **No inline editing** - Can't modify plan before applying + +### 3.4 Memory System + +**Current:** +- Post-level memory stored in `_wpaw_memory` meta +- Contains: summary, last_prompt, last_intent +- Chat history persisted per post + +**Gaps:** +1. **No global memory** - Can't learn user preferences across posts +2. **No style guide storage** - Can't save writing style preferences +3. **No context from other posts** - Can't reference previous work + +--- + +## Part 4: Agentic Gaps & Recommendations + +### 4.1 Critical Missing Features (High Priority) + +#### 4.1.1 Undo/Redo for AI Changes +**Problem:** No way to revert AI changes without manual Cmd+Z +**Solution:** +```javascript +// Add undo stack for AI operations +const aiUndoStack = []; + +const executeWithUndo = (operation) => { + const snapshot = captureEditorState(); + aiUndoStack.push(snapshot); + operation(); +}; + +// UI: Add "Undo AI" button in timeline entries +``` + +#### 4.1.2 Diff View Before Apply +**Problem:** Users can't see what will change before it happens +**Solution:** +- Add "Preview Changes" mode +- Show side-by-side or inline diff +- GitHub-style green/red highlighting + +#### 4.1.3 Accept/Reject Workflow +**Problem:** Changes apply immediately with no approval +**Solution:** +``` +┌─────────────────────────────────────────┐ +│ AI suggests: Replace paragraph 3 │ +│ │ +│ Before: "The old content..." │ +│ After: "The new content..." │ +│ │ +│ [Accept] [Reject] [Edit] [Skip] │ +└─────────────────────────────────────────┘ +``` + +### 4.2 Agentic Enhancements (Medium Priority) + +#### 4.2.1 Autonomous Multi-Step Execution +**Current:** User must approve each step +**Agentic:** Agent completes entire task autonomously + +**Recommendation:** Add "Full Auto" mode +```javascript +const agentModes = { + 'supervised': 'Approve each change', // Current + 'semi-auto': 'Approve plan, auto-execute', // New + 'full-auto': 'Complete task autonomously' // New (advanced) +}; +``` + +#### 4.2.2 Agent Memory & Learning +**Current:** No learning across sessions +**Agentic:** Remember user preferences, writing style + +**Recommendation:** +```php +// Global user preferences in wp_usermeta +update_user_meta($user_id, '_wpaw_preferences', [ + 'tone' => 'professional', + 'avoid_words' => ['leverage', 'synergy'], + 'preferred_length' => 'medium', + 'always_include' => ['code examples'], +]); +``` + +#### 4.2.3 Context-Aware Suggestions +**Current:** Agent only responds to commands +**Agentic:** Agent proactively suggests improvements + +**Recommendation:** +- Analyze article on idle +- Suggest improvements in sidebar +- "I noticed paragraph 3 could be clearer. Want me to refine it?" + +### 4.3 IDE-Like Features (Medium Priority) + +#### 4.3.1 Keyboard Shortcuts +| Shortcut | Action | +|----------|--------| +| `Cmd+Enter` | Send message | +| `Cmd+Shift+P` | Command palette | +| `Cmd+/` | Quick refine selected block | +| `Cmd+G` | Generate from selection | +| `Escape` | Cancel current operation | + +#### 4.3.2 Command Palette +``` +┌─────────────────────────────────────────┐ +│ > _ │ +├─────────────────────────────────────────┤ +│ 📝 Generate article from prompt │ +│ ✏️ Refine selected block │ +│ 📋 Create outline only │ +│ 🔄 Regenerate current section │ +│ 🌐 Enable web search │ +│ ⚙️ Open settings │ +└─────────────────────────────────────────┘ +``` + +#### 4.3.3 Block Outline Panel +``` +┌─────────────────────────────────────────┐ +│ ARTICLE STRUCTURE │ +├─────────────────────────────────────────┤ +│ ▼ Introduction │ +│ ├─ paragraph-1 │ +│ └─ paragraph-2 │ +│ ▼ Getting Started │ +│ ├─ heading-2 │ +│ ├─ paragraph-3 │ +│ └─ code-1 │ +│ ▼ Advanced Usage │ +│ ├─ heading-3 │ +│ └─ list-1 │ +└─────────────────────────────────────────┘ +``` + +### 4.4 UX Improvements (Lower Priority) + +#### 4.4.1 Inline Refinement +**Current:** Must use sidebar for all refinements +**Improvement:** Click block → inline popover with quick actions + +#### 4.4.2 Streaming Preview +**Current:** Content appears in editor directly +**Improvement:** Show in preview pane first, then "Apply All" + +#### 4.4.3 Smart Suggestions Bar +Show contextual actions based on selection: +``` +┌─────────────────────────────────────────┐ +│ 💡 Make concise │ 🔄 Rephrase │ 📝 Expand │ +└─────────────────────────────────────────┘ +``` + +--- + +## Part 5: Technical Debt & Code Quality + +### 5.1 Identified Issues + +1. **`sidebar.js` is 4,359 lines** - Should be split into modules +2. **Mixed concerns** - UI, state, API calls in same file +3. **No TypeScript** - Type safety would prevent bugs +4. **Hardcoded strings** - Should use i18n throughout + +### 5.2 Recommendations + +1. **Modularize sidebar.js:** + - `hooks/useChat.js` + - `hooks/usePlan.js` + - `hooks/useRefinement.js` + - `components/ChatTab.js` + - `components/ConfigTab.js` + - `utils/blockHelpers.js` + +2. **Add error boundaries** - React error boundaries for graceful failures + +3. **Implement proper state management** - Consider Redux or Zustand + +--- + +## Part 6: Prioritized Action Items + +### Tier 1: Critical (Do First) +| Item | Effort | Impact | Status | +|------|--------|--------|--------| +| Add Undo for AI changes | Medium | High | ❌ Not Implemented | +| Add Accept/Reject workflow | Medium | High | ✅ Implemented (Apply/Cancel buttons) | +| Add Cmd+Enter to send | Low | Medium | ✅ Implemented (line 2326 sidebar.js) | +| Add diff preview for edit plans | Medium | High | ✅ Implemented (edit_plan type with before/after) | + +> **Note:** Cross-verified on Jan 21, 2026. Only **Undo for AI changes** remains to be implemented in Tier 1. + +### Tier 2: Important (Do Next) +| Item | Effort | Impact | +|------|--------|--------| +| Command palette (Cmd+Shift+P) | Medium | High | +| Per-action accept/reject in plans | Medium | Medium | +| Block outline panel | Medium | Medium | +| Global user preferences | Low | Medium | + +### Tier 3: Nice to Have +| Item | Effort | Impact | +|------|--------|--------| +| Inline refinement popover | High | Medium | +| Streaming preview pane | High | Medium | +| Smart suggestions bar | Medium | Low | +| Full-auto mode | High | Low | + +--- + +## Part 7: Conclusion + +### What's Already Agentic ✅ +1. **@mention system** - Best-in-class block targeting +2. **Slash commands** - IDE-like quick actions +3. **Clarification quiz** - Proactive context gathering +4. **Edit plan preview** - Shows intent before action +5. **Section tracking** - Maintains document structure + +### What's Missing for True Agentic ❌ +1. **Approval workflow** - Changes should be reviewable +2. **Undo/history** - Need to revert AI mistakes +3. **Autonomous execution** - Agent should complete tasks independently +4. **Learning/memory** - Should improve over time +5. **Keyboard-first UX** - Power users need shortcuts + +### Final Recommendation + +The plugin has **strong agentic bones** but needs the **safety net** features that make IDEs trustworthy: + +1. **Immediate wins:** Keyboard shortcuts + Undo button +2. **Medium-term:** Accept/Reject workflow + Command palette +3. **Long-term:** Autonomous mode + Learning system + +The goal should be: *"I can trust this agent to make changes because I can always review, approve, or revert."* + +--- + +## Part 8: Proactive AI Suggestions (NEW REQUIREMENT) + +### 8.1 Current State +**Problem:** Agent only responds to commands - purely reactive. + +### 8.2 Target State +**Goal:** Agent proactively analyzes content and suggests improvements. + +### 8.3 Implementation Specification + +#### 8.3.1 Idle Analysis Trigger +```javascript +// Trigger analysis after user stops editing for N seconds +const IDLE_THRESHOLD_MS = 5000; // 5 seconds +let idleTimer = null; + +const startIdleAnalysis = () => { + clearTimeout(idleTimer); + idleTimer = setTimeout(() => { + analyzeArticleForSuggestions(); + }, IDLE_THRESHOLD_MS); +}; + +// Hook into editor changes +wp.data.subscribe(() => { + startIdleAnalysis(); +}); +``` + +#### 8.3.2 Suggestion Types +| Category | Example Suggestion | +|----------|--------------------| +| **Clarity** | "Paragraph 3 could be clearer. Want me to simplify it?" | +| **Flow** | "The transition between sections 2 and 3 feels abrupt." | +| **Depth** | "This section could use more examples or data." | +| **SEO** | "Consider adding keyword 'X' to heading 2." | +| **Structure** | "This article could benefit from a summary section." | +| **Engagement** | "Consider adding a question to engage readers here." | + +#### 8.3.3 UI Component +``` +┌─────────────────────────────────────────────────┐ +│ 💡 Suggestion │ +│ │ +│ "I noticed paragraph 3 could be clearer. │ +│ Want me to refine it?" │ +│ │ +│ [Apply] [Dismiss] [Don't show again] │ +└─────────────────────────────────────────────────┘ +``` + +#### 8.3.4 Backend Endpoint +```php +// New REST endpoint: /analyze-for-suggestions +public function analyze_for_suggestions() { + $blocks = $this->get_all_blocks(); + $prompt = "Analyze this article and suggest 1-3 improvements..."; + // Return structured suggestions +} +``` + +#### 8.3.5 User Preferences +- Toggle: "Enable proactive suggestions" +- Frequency: "Aggressive / Balanced / Minimal" +- Categories: Checkboxes for which suggestion types to show + +--- + +## Part 9: SEO Specialist Capabilities (NEW REQUIREMENT) + +### 9.1 Vision +Agentic Writer should not only write well but write **SEO-optimized content** that ranks. + +### 9.2 SEO Feature Matrix + +| Feature | Description | Priority | +|---------|-------------|----------| +| **Keyword Analysis** | Analyze target keyword, suggest density | High | +| **Keyword Placement** | Ensure keyword in title, H1, first paragraph | High | +| **Heading Structure** | Validate H1→H2→H3 hierarchy | High | +| **Meta Generation** | Auto-generate meta title & description | High | +| **Readability Score** | Flesch-Kincaid or similar | Medium | +| **Internal Linking** | Suggest links to other posts | Medium | +| **Image Alt Text** | Auto-generate SEO-friendly alt text | Medium | +| **Schema Markup** | Suggest FAQ, HowTo, Article schema | Medium | +| **Competitor Analysis** | Compare with top-ranking articles | Low | +| **SERP Preview** | Show how it will appear in Google | Low | + +### 9.3 SEO Workflow Integration + +#### 9.3.1 Pre-Writing Phase +``` +┌─────────────────────────────────────────────────┐ +│ 🎯 SEO Setup │ +│ │ +│ Target Keyword: [_______________] │ +│ Secondary Keywords: [_______________] │ +│ │ +│ ☑ Analyze competition before writing │ +│ ☑ Include keyword in title │ +│ ☑ Suggest internal links │ +│ │ +│ [Analyze Competition] [Skip to Writing] │ +└─────────────────────────────────────────────────┘ +``` + +#### 9.3.2 During Writing (Real-time) +- **Keyword density indicator** in sidebar +- **Heading structure validator** +- **Reading time & word count** +- **Readability score (live)** + +#### 9.3.3 Post-Writing (SEO Audit) +``` +┌─────────────────────────────────────────────────┐ +│ 📊 SEO Score: 78/100 │ +│ │ +│ ✅ Keyword in title │ +│ ✅ Keyword in first paragraph │ +│ ⚠️ Keyword density low (0.8%, target 1-2%) │ +│ ❌ Missing meta description │ +│ ❌ No internal links found │ +│ ✅ Proper heading hierarchy │ +│ ⚠️ Images missing alt text (2 of 3) │ +│ │ +│ [Fix All Issues] [Generate Meta] [Add Links] │ +└─────────────────────────────────────────────────┘ +``` + +### 9.4 SEO-Aware System Prompts + +Modify the plan generation prompt to include SEO considerations: + +``` +You are an SEO-optimized content writer. When creating content: + +1. KEYWORD PLACEMENT: + - Include target keyword in H1 and first 100 words + - Use keyword naturally 1-2% density + - Include semantic variations + +2. STRUCTURE: + - Use proper heading hierarchy (H1→H2→H3) + - Include table of contents for long articles + - Use bullet points and numbered lists + - Keep paragraphs under 150 words + +3. ENGAGEMENT: + - Start with a hook + - Use questions to engage readers + - Include actionable takeaways + +4. TECHNICAL: + - Suggest descriptive image alt text + - Recommend internal link opportunities + - Optimize for featured snippets where applicable +``` + +### 9.5 Post Config Additions + +```javascript +const seoConfig = { + target_keyword: '', + secondary_keywords: [], + enable_seo_mode: true, + keyword_density_target: 1.5, // percentage + min_word_count: 1500, + include_meta: true, + include_schema: false, + internal_linking: true, +}; +``` + +### 9.6 New REST Endpoints + +| Endpoint | Purpose | +|----------|--------| +| `/seo/analyze-keyword` | Get keyword difficulty, volume | +| `/seo/audit-content` | Full SEO audit of current content | +| `/seo/generate-meta` | Generate meta title/description | +| `/seo/suggest-links` | Find internal linking opportunities | +| `/seo/competitor-analysis` | Analyze top-ranking content | + +--- + +## Part 10: AI Model Recommendations (NEW REQUIREMENT) + +### 10.1 Purpose +Help users choose the **cheapest AND best** models for agentic workflows. + +### 10.2 Model Tiers by Use Case + +#### 10.2.1 Planning Phase (Fast, Cheap) +| Model | Cost (per 1M tokens) | Speed | Quality | Recommendation | +|-------|---------------------|-------|---------|----------------| +| `google/gemini-2.0-flash-exp` | ~$0.10 in / $0.40 out | ⚡ Very Fast | Good | **Best Value** | +| `google/gemini-flash-1.5` | ~$0.075 in / $0.30 out | ⚡ Very Fast | Good | Budget Option | +| `anthropic/claude-3-haiku` | ~$0.25 in / $1.25 out | Fast | Good | Reliable | +| `openai/gpt-4o-mini` | ~$0.15 in / $0.60 out | Fast | Good | Alternative | + +#### 10.2.2 Execution Phase (Quality Critical) +| Model | Cost (per 1M tokens) | Speed | Quality | Recommendation | +|-------|---------------------|-------|---------|----------------| +| `anthropic/claude-sonnet-4` | ~$3 in / $15 out | Medium | Excellent | **Best for Writing** | +| `anthropic/claude-3.5-sonnet` | ~$3 in / $15 out | Medium | Excellent | Proven Quality | +| `openai/gpt-4o` | ~$2.50 in / $10 out | Medium | Excellent | Alternative | +| `google/gemini-pro-1.5` | ~$1.25 in / $5 out | Medium | Very Good | Cost-Conscious | + +#### 10.2.3 Image Generation +| Model | Cost (per image) | Speed | Quality | Recommendation | +|-------|-----------------|-------|---------|----------------| +| `black-forest-labs/flux-schnell` | ~$0.003 | ⚡ Very Fast | Good | **Best Value** | +| `black-forest-labs/flux-1.1-pro` | ~$0.04 | Fast | Excellent | Premium | +| `openai/dall-e-3` | ~$0.04-0.08 | Medium | Excellent | Alternative | + +### 10.3 Recommended Configurations + +#### 10.3.1 Budget-Conscious Setup (~$0.05 per article) +``` +Planning Model: google/gemini-2.0-flash-exp +Execution Model: google/gemini-pro-1.5 +Image Model: black-forest-labs/flux-schnell +``` + +#### 10.3.2 Balanced Setup (~$0.15 per article) +``` +Planning Model: google/gemini-2.0-flash-exp +Execution Model: anthropic/claude-3.5-sonnet +Image Model: black-forest-labs/flux-schnell +``` + +#### 10.3.3 Premium Quality Setup (~$0.30+ per article) +``` +Planning Model: anthropic/claude-3-haiku +Execution Model: anthropic/claude-sonnet-4 +Image Model: black-forest-labs/flux-1.1-pro +``` + +### 10.4 Model Selection UI + +``` +┌─────────────────────────────────────────────────┐ +│ 🤖 Model Configuration │ +│ │ +│ Preset: [Budget ▾] [Balanced ▾] [Premium ▾] │ +│ │ +│ Planning Model: │ +│ [google/gemini-2.0-flash-exp ▾] │ +│ 💰 ~$0.10/1M tokens • ⚡ Fast │ +│ │ +│ Execution Model: │ +│ [anthropic/claude-sonnet-4 ▾] │ +│ 💰 ~$3/1M tokens • ✨ Best Quality │ +│ │ +│ Image Model: │ +│ [black-forest-labs/flux-schnell ▾] │ +│ 💰 ~$0.003/image • ⚡ Fast │ +│ │ +│ Estimated cost per article: ~$0.15 │ +└─────────────────────────────────────────────────┘ +``` + +### 10.5 Smart Model Suggestions + +The plugin should analyze usage patterns and suggest: + +1. **If budget is tight:** + > "You've spent $45 this month. Switch to Budget preset to save ~60%." + +2. **If quality issues detected:** + > "Multiple refinements on recent articles. Consider upgrading execution model." + +3. **If speed is priority:** + > "For faster generation, switch planning to Gemini Flash." + +--- + +## Part 11: Updated Roadmap + +### Phase 1: Complete Tier 1 ✅ COMPLETED +- [x] Cmd+Enter to send +- [x] Accept/Reject workflow (Apply/Cancel) +- [x] Diff preview for edit plans +- [x] **Undo for AI changes** (sidebar.js: aiUndoStack, pushUndoSnapshot, undoLastAiOperation) +- [x] **Budget tracker enhanced** (Cost tab with refresh, remaining display, warnings) +- [x] **Settings page revamp** (Modern card-based UI with preset configurations) + +### Phase 2: SEO Foundation ✅ COMPLETED +- [x] Add SEO config fields (focus keyword, secondary keywords, meta description) +- [x] Integrate SEO considerations into system prompts (build_seo_context) +- [x] Add keyword density indicator (in SEO audit) +- [x] Add SEO audit endpoint (/seo-audit/{post_id}) +- [x] Add meta generation (/generate-meta with AI) + +### Phase 3: Proactive Suggestions (2-3 weeks) +- [ ] Implement idle detection +- [ ] Create suggestion analysis endpoint +- [ ] Add suggestion UI component +- [ ] Add user preferences for suggestions + +### Phase 4: Model Recommendations ✅ COMPLETED +- [x] Add preset configurations (Budget, Balanced, Premium) +- [x] Add model selection UI with cost estimates +- [ ] Add smart suggestions based on usage + +### Phase 5: Tier 2 Features (3-4 weeks) +- [ ] Command palette (Cmd+Shift+P) +- [ ] Per-action accept/reject in plans +- [ ] Block outline panel +- [ ] Global user preferences + +--- + +**Report Updated:** January 22, 2026 +**Implementation Status:** Phase 1, Phase 2 (SEO Foundation) & Phase 4 completed. Ready for Phase 3 (Proactive Suggestions). diff --git a/AGENTIC_CONTEXT_STRATEGY.md b/AGENTIC_CONTEXT_STRATEGY.md new file mode 100644 index 0000000..7e7e27b --- /dev/null +++ b/AGENTIC_CONTEXT_STRATEGY.md @@ -0,0 +1,786 @@ +# Agentic Context Management Strategy + +**Date:** January 25, 2026 +**Version:** 0.1.3+ +**Purpose:** AI-powered context management for multilingual, intelligent user experience + +--- + +## 🎯 Core Philosophy: Let AI Handle AI Context + +### **The Problem with Hardcoded Solutions** + +**Previous Approach (FLAWED):** +```javascript +// ❌ English-only, brittle, not scalable +if (content.includes('outline') || content.includes('structure')) { + return 'create_outline'; +} +``` + +**Issues:** +- ❌ Only works in English +- ❌ Breaks in Indonesian, Arabic, Chinese, etc. +- ❌ Misses nuanced intent +- ❌ Requires constant maintenance +- ❌ Goes against "agentic" philosophy + +### **Agentic Principle** + +> **"If AI is smart enough to write articles, it's smart enough to manage its own context."** + +**New Approach:** +- ✅ Use AI to summarize chat history +- ✅ Use AI to detect user intent +- ✅ Language-agnostic (works in any language) +- ✅ Adapts to context automatically +- ✅ True "agentic" experience + +--- + +## 💰 Cost Analysis: AI-Powered Context Management + +### **Your Cost Reference** + +``` +Action: meta_description +Model: deepseek-chat-v3-032 +Tokens: 510 +Cost: $0.0001 +``` + +**This is EXTREMELY cheap!** Let's use this model for context operations. + +### **Proposed Actions** + +#### **1. Action: `summarize_context`** + +**Purpose:** Condense long chat history into key points + +**Input:** +```json +{ + "action": "summarize_context", + "chat_history": [ + {"role": "user", "content": "Saya ingin menulis tentang keamanan WordPress"}, + {"role": "assistant", "content": "[Long response in Indonesian...]"}, + {"role": "user", "content": "Fokus pada plugin vulnerabilities saja"}, + {"role": "assistant", "content": "[Detailed plugin security response...]"} + ] +} +``` + +**Prompt:** +``` +Summarize this conversation into key points that capture the user's intent and requirements. +Focus on: +- Main topic +- Specific focus areas +- Rejected/excluded topics +- User preferences (tone, audience, etc.) + +Keep the summary concise (max 200 words) but preserve critical context. +Write in the same language as the conversation. + +Output format: +TOPIC: [main topic] +FOCUS: [what to include] +EXCLUDE: [what to avoid] +PREFERENCES: [any specific requirements] +``` + +**Expected Output:** +``` +TOPIC: WordPress security +FOCUS: Plugin vulnerabilities only +EXCLUDE: Performance optimization, backup strategies (user rejected these) +PREFERENCES: Technical audience, detailed explanations +``` + +**Cost Estimate:** +- Input: 4,000 tokens (long chat history) +- Output: 100 tokens (summary) +- Model: deepseek-chat-v3-032 +- **Cost: ~$0.0001 per summarization** + +**When to Use:** +- Chat history > 6 messages +- Before generating outline +- Before executing article + +--- + +#### **2. Action: `detect_intent`** + +**Purpose:** Understand what user wants to do next + +**Input:** +```json +{ + "action": "detect_intent", + "last_message": "Baiklah, sekarang buatkan outline-nya", + "has_plan": false, + "current_mode": "chat" +} +``` + +**Prompt:** +``` +Based on the user's message, determine their intent. Choose ONE: + +1. "create_outline" - User wants to create an article outline/structure +2. "start_writing" - User wants to write the full article +3. "refine_content" - User wants to improve existing content +4. "continue_chat" - User wants to continue discussing/exploring +5. "clarify" - User is asking questions or needs clarification + +Consider: +- The user's explicit request +- Whether they have an outline already (has_plan: {has_plan}) +- Current mode (current_mode: {current_mode}) + +Respond with ONLY the intent code (e.g., "create_outline"). +``` + +**Expected Output:** +``` +create_outline +``` + +**Cost Estimate:** +- Input: 100 tokens (last message + context) +- Output: 5 tokens (intent code) +- Model: deepseek-chat-v3-032 +- **Cost: ~$0.00002 per detection** + +**When to Use:** +- After every user message in Chat mode +- To show contextual action buttons +- To auto-suggest next steps + +--- + +## 📊 Cost Comparison: Full History vs AI-Powered + +### **Scenario: 5 Agent + 4 Human Messages** + +| Approach | Input Tokens | Output Tokens | Cost per Request | Quality | Language Support | +|----------|--------------|---------------|------------------|---------|------------------| +| **Full History** | 4,365 | 0 | $0.013 (Claude) | ✅ Best | ✅ All | +| **AI Summarization** | 100 (summary) | 0 | $0.003 (Claude) + $0.0001 (summary) | ✅ Good | ✅ All | +| **Hardcoded Pruning** | 1,800 | 0 | $0.005 (Claude) | ⚠️ Fair | ❌ English only | +| **No Context** | 0 | 0 | $0.000 | ❌ Poor | ✅ All | + +### **Cost Breakdown for 100 Articles/Month** + +**Full History:** +- Planning: 100 × $0.00033 = $0.033 +- Execution: 100 × $0.013 = $1.30 +- **Total: $1.33/month** + +**AI Summarization:** +- Summarization: 100 × $0.0001 = $0.01 +- Planning: 100 × $0.00008 = $0.008 +- Execution: 100 × $0.003 = $0.30 +- **Total: $0.32/month** +- **Savings: $1.01/month (76% reduction)** + +**Intent Detection:** +- Per message: $0.00002 +- Average 10 messages per article: 100 × 10 × $0.00002 = $0.02 +- **Total: $0.02/month (negligible)** + +--- + +## 🎯 The Big Picture: Agentic Experience + +### **User Journey Analysis** + +**Current Flow (Fragmented):** +``` +1. User opens editor +2. User manually switches to Chat mode +3. User types message +4. Agent responds +5. User types more +6. Agent responds +7. User manually switches to Planning mode +8. User types "create outline" +9. Outline generated +10. User manually clicks "Start Writing" +11. Article generated +``` + +**Problems:** +- Too many manual mode switches +- User must know when to switch +- No guidance on next steps +- Friction in workflow + +--- + +### **Proposed Agentic Flow (Seamless):** + +``` +1. User opens editor (any mode) +2. User types: "Saya ingin menulis tentang keamanan WordPress" +3. Agent responds with suggestions +4. User types: "Fokus pada plugin vulnerabilities" +5. Agent responds with refined ideas +6. 💡 UI shows: [📝 Ready to create outline?] (AI-detected intent) +7. User clicks button (or types "yes" or "buatkan outline") +8. ✨ AI summarizes chat history (0.1 seconds) +9. Outline generated with clean context +10. 💡 UI shows: [✍️ Start Writing] (auto-suggested) +11. User clicks +12. Article generated +``` + +**Improvements:** +- ✅ No manual mode switching needed +- ✅ AI suggests next steps proactively +- ✅ Context automatically optimized +- ✅ Smooth, guided experience +- ✅ Works in any language + +--- + +## 🔧 Implementation Design + +### **Backend: New Actions** + +```php +/** + * Handle context summarization request. + * + * @param WP_REST_Request $request REST request. + * @return WP_REST_Response|WP_Error Response. + */ +public function handle_summarize_context( $request ) { + $params = $request->get_json_params(); + $chat_history = $params['chatHistory'] ?? array(); + + if ( empty( $chat_history ) || count( $chat_history ) < 4 ) { + // No need to summarize short history + return new WP_REST_Response( + array( + 'summary' => '', + 'use_full_history' => true, + ), + 200 + ); + } + + // Build summarization prompt + $history_text = ''; + foreach ( $chat_history as $msg ) { + $role = ucfirst( $msg['role'] ?? 'Unknown' ); + $content = $msg['content'] ?? ''; + $history_text .= "{$role}: {$content}\n\n"; + } + + $prompt = "Summarize this conversation into key points that capture the user's intent and requirements. + +Focus on: +- Main topic +- Specific focus areas +- Rejected/excluded topics +- User preferences (tone, audience, etc.) + +Keep the summary concise (max 200 words) but preserve critical context. +Write in the same language as the conversation. + +Output format: +TOPIC: [main topic] +FOCUS: [what to include] +EXCLUDE: [what to avoid] +PREFERENCES: [any specific requirements] + +Conversation: +{$history_text}"; + + $provider = WP_Agentic_Writer_OpenRouter_Provider::get_instance(); + $messages = array( + array( + 'role' => 'user', + 'content' => $prompt, + ), + ); + + // Use cheap model for summarization + $response = $provider->chat( $messages, array(), 'summarize' ); + + if ( is_wp_error( $response ) ) { + return $response; + } + + // Track cost + do_action( + 'wp_aw_after_api_request', + $params['postId'] ?? 0, + $response['model'] ?? '', + 'summarize_context', + $response['input_tokens'] ?? 0, + $response['output_tokens'] ?? 0, + $response['cost'] ?? 0 + ); + + return new WP_REST_Response( + array( + 'summary' => $response['content'] ?? '', + 'use_full_history' => false, + 'cost' => $response['cost'] ?? 0, + ), + 200 + ); +} + +/** + * Handle intent detection request. + * + * @param WP_REST_Request $request REST request. + * @return WP_REST_Response|WP_Error Response. + */ +public function handle_detect_intent( $request ) { + $params = $request->get_json_params(); + $last_message = $params['lastMessage'] ?? ''; + $has_plan = $params['hasPlan'] ?? false; + $current_mode = $params['currentMode'] ?? 'chat'; + + if ( empty( $last_message ) ) { + return new WP_REST_Response( + array( 'intent' => 'continue_chat' ), + 200 + ); + } + + $prompt = "Based on the user's message, determine their intent. Choose ONE: + +1. \"create_outline\" - User wants to create an article outline/structure +2. \"start_writing\" - User wants to write the full article +3. \"refine_content\" - User wants to improve existing content +4. \"continue_chat\" - User wants to continue discussing/exploring +5. \"clarify\" - User is asking questions or needs clarification + +Consider: +- The user's explicit request +- Whether they have an outline already (has_plan: " . ( $has_plan ? 'true' : 'false' ) . ") +- Current mode (current_mode: {$current_mode}) + +User's message: \"{$last_message}\" + +Respond with ONLY the intent code (e.g., \"create_outline\")."; + + $provider = WP_Agentic_Writer_OpenRouter_Provider::get_instance(); + $messages = array( + array( + 'role' => 'user', + 'content' => $prompt, + ), + ); + + $response = $provider->chat( $messages, array(), 'intent_detection' ); + + if ( is_wp_error( $response ) ) { + return $response; + } + + // Track cost + do_action( + 'wp_aw_after_api_request', + $params['postId'] ?? 0, + $response['model'] ?? '', + 'detect_intent', + $response['input_tokens'] ?? 0, + $response['output_tokens'] ?? 0, + $response['cost'] ?? 0 + ); + + $intent = trim( strtolower( $response['content'] ?? 'continue_chat' ) ); + + return new WP_REST_Response( + array( + 'intent' => $intent, + 'cost' => $response['cost'] ?? 0, + ), + 200 + ); +} +``` + +--- + +### **Frontend: Agentic UX** + +```javascript +// Auto-detect intent after each message +const handleMessageSent = async (userMessage) => { + // Send message to chat + const chatResponse = await sendChatMessage(userMessage); + + // Detect intent in background + const intentResponse = await fetch('/wp-json/wp-agentic-writer/v1/detect-intent', { + method: 'POST', + body: JSON.stringify({ + lastMessage: userMessage, + hasPlan: !!currentPlan, + currentMode: agentMode, + postId: postId + }) + }); + + const { intent } = await intentResponse.json(); + + // Show contextual action based on intent + setDetectedIntent(intent); + showContextualAction(intent); +}; + +// Render contextual action buttons +const renderContextualAction = () => { + if (!detectedIntent) return null; + + switch (detectedIntent) { + case 'create_outline': + return ( +
💡 Ready to create an outline?
+ +⚠️ You need an outline first
+ +💡 Ready to write the article?
+ +💡 Use @block to refine specific sections
++ v · + +
+| + | + | + | + | + | + | + | + |
|---|
${escapeHtml(record.created_at)}' . wp_kses_post( $args['description'] ) . '
'; + } +} + +public function clarification_quiz_section_callback() { + echo '' . __( 'Configure when and how the clarification quiz appears to gather context for better article generation.', 'wp-agentic-writer' ) . '
'; +} +``` + +**Hook Integration:** +Add to `__construct()` or settings initialization: +```php +add_action( 'admin_init', array( $this, 'add_clarification_quiz_settings' ), 20 ); +``` + +--- + +## Phase 2: Update Clarity Check System Prompt + +### File: `includes/class-gutenberg-sidebar.php` + +**Location:** Lines 1195-1231 (check_clarity method) + +**Replace the current system prompt with:** + +```php +$required_categories = get_option( 'required_context_categories', array( + 'target_outcome', + 'target_audience', + 'tone', + 'content_depth', + 'expertise_level', + 'content_type', + 'pov' +) ); + +$threshold = get_option( 'clarity_confidence_threshold', '0.6' ); +$enabled = get_option( 'enable_clarification_quiz', true ); + +// If quiz is disabled, always return clear +if ( ! $enabled ) { + return new WP_REST_Response( + array( + 'result' => array( + 'is_clear' => true, + 'confidence' => 1.0, + 'questions' => array() + ), + ), + 200 + ); +} + +$system_prompt = "You are an expert editor who determines if an article request has sufficient context to write effectively. + +Evaluate the user's request and determine which context categories are clear: + +CATEGORIES TO EVALUATE: +1. target_outcome - What should this content achieve? (education/marketing/sales/entertainment/brand_awareness) +2. target_audience - Who is reading this? (demographics, role, knowledge level) +3. tone - How should we sound? (formal/casual/technical/friendly/professional/conversational) +4. content_depth - How comprehensive? (quick_overview/standard_guide/detailed_analysis/comprehensive) +5. expertise_level - Reader's knowledge? (beginner/intermediate/advanced/expert) +6. content_type - What format? (tutorial/how_to/opinion/comparison/listicle/case_study/news_analysis) +7. pov - Whose perspective? (first_person/third_person/expert_voice/neutral) + +For each MISSING category, generate a clarifying question using PREDEFINED OPTIONS. +Use 'single_choice' or 'multiple_choice' types - NEVER 'open_text'. + +QUESTION STRUCTURE: +{ + 'id': 'q1', + 'category': 'target_outcome', + 'question': 'What is the primary goal of this content?', + 'type': 'single_choice', + 'options': [ + { 'value': 'Education - Teach something new', 'default': true }, + { 'value': 'Marketing - Promote a product/service', 'default': false }, + { 'value': 'Sales - Drive conversions/signups', 'default': false }, + { 'value': 'Entertainment - Engage and entertain', 'default': false }, + { 'value': 'Brand Awareness - Build authority/trust', 'default': false } + ] +} + +CONFIDENCE CALCULATION: +- Start at 100% (1.0) +- Subtract 15% for each missing required category +- If confidence < {$threshold}, generate questions for ALL missing categories + +Return ONLY valid JSON with this structure: +{ + 'is_clear': true/false, + 'confidence': 0.0-1.0, + 'missing_categories': ['category1', 'category2'], + 'questions': [ ... ] +} + +No markdown, no explanation - just JSON."; + +$messages = array( + array( + 'role' => 'system', + 'content' => $system_prompt, + ), + array( + 'role' => 'user', + 'content' => "Topic: {$topic}\n\nRequired Categories: " . implode( ', ', $required_categories ) . "\n\nEvaluate this request and determine which context is missing.", + ), +); +``` + +--- + +## Phase 3: Improve Fallback Behavior + +### File: `includes/class-gutenberg-sidebar.php` + +**Location:** Around lines 1603-1615 + +**Add new helper method:** + +```php +/** + * Get default clarification questions when AI fails + * + * @since 0.1.0 + * @param string $topic User's topic. + * @return array Clarification result with default questions. + */ +private function get_default_clarification_questions( $topic ) { + $required_categories = get_option( 'required_context_categories', array( + 'target_outcome', + 'target_audience', + 'tone', + 'content_depth', + 'expertise_level', + 'content_type', + 'pov' + ) ); + + $questions = array(); + $question_id = 1; + + $question_templates = array( + 'target_outcome' => array( + 'category' => 'target_outcome', + 'question' => 'What is the primary goal of this content?', + 'type' => 'single_choice', + 'options' => array( + array( 'value' => 'Education - Teach something new', 'default' => true ), + array( 'value' => 'Marketing - Promote a product/service', 'default' => false ), + array( 'value' => 'Sales - Drive conversions', 'default' => false ), + array( 'value' => 'Entertainment - Engage readers', 'default' => false ), + array( 'value' => 'Brand Awareness - Build authority', 'default' => false ), + ) + ), + 'target_audience' => array( + 'category' => 'target_audience', + 'question' => 'Who is the primary audience for this content?', + 'type' => 'single_choice', + 'options' => array( + array( 'value' => 'General public / Beginners', 'default' => true ), + array( 'value' => 'Professionals in the field', 'default' => false ), + array( 'value' => 'Potential customers', 'default' => false ), + array( 'value' => 'Existing customers/users', 'default' => false ), + array( 'value' => 'Industry peers / Experts', 'default' => false ), + ) + ), + 'tone' => array( + 'category' => 'tone', + 'question' => 'What tone should this content have?', + 'type' => 'single_choice', + 'options' => array( + array( 'value' => 'Professional & Authoritative', 'default' => true ), + array( 'value' => 'Friendly & Conversational', 'default' => false ), + array( 'value' => 'Technical & Detailed', 'default' => false ), + array( 'value' => 'Casual & Entertaining', 'default' => false ), + array( 'value' => 'Formal & Academic', 'default' => false ), + ) + ), + 'content_depth' => array( + 'category' => 'content_depth', + 'question' => 'How comprehensive should this content be?', + 'type' => 'single_choice', + 'options' => array( + array( 'value' => 'Quick overview (500-800 words)', 'default' => false ), + array( 'value' => 'Standard guide (800-1500 words)', 'default' => true ), + array( 'value' => 'Detailed analysis (1500-2500 words)', 'default' => false ), + array( 'value' => 'Comprehensive deep-dive (2500+ words)', 'default' => false ), + ) + ), + 'expertise_level' => array( + 'category' => 'expertise_level', + 'question' => 'What is the target audience\'s expertise level?', + 'type' => 'single_choice', + 'options' => array( + array( 'value' => 'Beginner - No prior knowledge', 'default' => true ), + array( 'value' => 'Intermediate - Basic understanding', 'default' => false ), + array( 'value' => 'Advanced - Deep technical knowledge', 'default' => false ), + array( 'value' => 'Expert - Industry professional', 'default' => false ), + ) + ), + 'content_type' => array( + 'category' => 'content_type', + 'question' => 'What type of content works best for this topic?', + 'type' => 'single_choice', + 'options' => array( + array( 'value' => 'Tutorial / How-to guide', 'default' => true ), + array( 'value' => 'Opinion / Commentary', 'default' => false ), + array( 'value' => 'Comparison / Review', 'default' => false ), + array( 'value' => 'Listicle / Tips', 'default' => false ), + array( 'value' => 'Case study', 'default' => false ), + array( 'value' => 'News analysis', 'default' => false ), + ) + ), + 'pov' => array( + 'category' => 'pov', + 'question' => 'From what perspective should this be written?', + 'type' => 'single_choice', + 'options' => array( + array( 'value' => 'Third person (objective, "it", "they")', 'default' => true ), + array( 'value' => 'First person (personal, "I", "my")', 'default' => false ), + array( 'value' => 'Expert voice (authoritative, experienced)', 'default' => false ), + array( 'value' => 'Neutral / Unbiased', 'default' => false ), + ) + ), + ); + + foreach ( $required_categories as $category ) { + if ( isset( $question_templates[ $category ] ) ) { + $q = $question_templates[ $category ]; + $q['id'] = 'q' . $question_id++; + $questions[] = $q; + } + } + + return array( + 'is_clear' => false, + 'confidence' => 0.0, + 'missing_categories' => $required_categories, + 'questions' => $questions + ); +} +``` + +**Update error handling (lines 1603-1615):** + +```php +if ( is_wp_error( $response ) ) { + // Log error + error_log( 'WP Agentic Writer: Clarity check API error - ' . $response->get_error_message() ); + // Use default questions instead of skipping + return $this->get_default_clarification_questions( $topic ); +} + +if ( null === $result ) { + // Log parse error + error_log( 'WP Agentic Writer: Failed to parse clarity check JSON' ); + // Use default questions instead of skipping + return $this->get_default_clarification_questions( $topic ); +} +``` + +--- + +## Phase 4: Update Frontend Quiz Display + +### File: `assets/js/sidebar.js` + +**Enhancement 1: Add category labels to questions** + +Find the question rendering code and add category display: + +```javascript +// Inside renderClarificationQuestion function +const categoryLabels = { + 'target_outcome': '🎯 Target Outcome', + 'target_audience': '👥 Target Audience', + 'tone': '🎨 Tone of Voice', + 'content_depth': '📏 Content Depth', + 'expertise_level': '📊 Expertise Level', + 'content_type': '📝 Content Type', + 'pov': '👁️ Point of View' +}; + +// Add category badge above question +if (question.category && categoryLabels[question.category]) { + html += 'Writing mode requires an outline to structure your article.
+ ++ Or switch to to discuss your ideas. +
+