wp-agentic-writer/CONTEXT_GAP_DIAGNOSTIC_REPORT.md

Context Gap Diagnostic Report

Date: January 30, 2026
Issue: Context lost during outline generation - Agent doesn't maintain conversation continuity
Severity: High - Core agentic behavior broken

---

Executive Summary

The agent loses context when generating outlines because:

1. Generic topic passed instead of extracting actual topic from conversation
2. Chat history truncated to last 10 messages (first message with core topic lost)
3. No topic extraction mechanism - system relies on recency, not importance
4. Recency bias - LLM sees recent refinements more than original intent

---

Conversation Flow Analysis

User's Conversation Timeline

Step	User Action	Agent Response	Context Status
1	Rich topic: "switch career usia 30+, AI, web design, vibe coding..."	Comprehensive response	FULL CONTEXT
2	"tambahkan vibe coding"	Added vibe coding section	FULL CONTEXT
3	"fokus opini, web design, tanpa coding"	Refined to web design focus	FULL CONTEXT
4	Click "Create Outline Now"	Generated outline about "AI Web Design"	CONTEXT LOST
5	User manually sets focus keyword, asks to redo	Regenerated with correct focus	RECOVERED (manually)

Where Context Was Lost

Step 4: "Create Outline Now" button click

The outline focused on "AI-Powered Web Design" instead of the broader "Switch Career Usia 30+" topic that was the user's original intent.

---

Root Cause Analysis

Defect #1: Generic Topic Parameter

Location: assets/js/sidebar.js:4534

body: JSON.stringify({
    topic: outlineMessage,  // "Create an outline based on our discussion"
    // ...
})

Problem: The topic variable is set to the literal string "Create an outline based on our discussion" instead of extracting the actual topic from the first user message.

Impact: The LLM receives a generic topic and must infer intent from chat history alone.

---

Defect #2: Chat History Truncation (.slice(-10))

Location: assets/js/sidebar.js:4543

chatHistory: messages.filter(m => m.role === 'user' || m.role === 'assistant').slice(-10),

Problem: Only the last 10 messages are sent to the backend. If the conversation has more than 10 exchanges, the first user message (which contains the core topic) is lost.

Your Conversation Analysis:

• Message 1: User's detailed topic request (CRITICAL - contains "switch career usia 30+")
• Message 2: Agent's comprehensive response
• Message 3: User adds "vibe coding"
• Message 4: Agent responds about vibe coding
• Message 5: User refines to "web design focus"
• Message 6: Agent responds about web design
• Message 7: User clicks "Create Outline Now" (adds another message)

With .slice(-10), the first message might still be included in this case, but the truncation creates fragility. The real issue is combined with Defect #1.

---

Defect #3: No Topic Extraction Mechanism

Location: includes/class-gutenberg-sidebar.php:1765

'content' => "Topic: {$topic}\n\nContext: {$context}{$chat_history_context}..."

Problem: The system doesn't extract the user's original topic/intent from the first message. It just appends chat history as context, but the LLM prompt structure puts emphasis on {$topic} which is generic.

What should happen:

1. Extract topic from first user message
2. Store as "primary topic" in post memory
3. Use primary topic in outline generation, not generic phrase

---

Defect #4: Recency Bias in LLM Processing

Problem: When the LLM sees the chat history, the most recent messages (about web design) appear at the end and have more weight than earlier messages about the broader topic.

Chat History Seen by LLM:

User: [switch career usia 30+ topic...] ← EARLY, less weight
Assistant: [comprehensive response...]
User: [add vibe coding...]
Assistant: [vibe coding response...]
User: [focus on web design...] ← RECENT, more weight
Assistant: [web design response...] ← MOST RECENT, highest weight
User: Create an outline based on our discussion

The LLM naturally focuses on the most recent topic (web design) rather than the original broader topic.

---

Defect #5: Memory System Doesn't Store Primary Topic

Location: includes/class-gutenberg-sidebar.php:4735-4748

private function update_post_memory( $post_id, $data ) {
    // Only stores: summary, last_prompt, last_intent
    // Does NOT store: primary_topic, original_intent, focus_keyword
}

Problem: The memory system stores last_prompt but not primary_topic. When generating an outline, there's no reference to what the user originally wanted.

---

Impact Analysis

Aspect	Impact
User Experience	Frustrating - user must manually correct agent's misunderstanding
Cost	Wasted API calls on incorrect outline generation
Trust	User loses confidence in agent's ability to understand context
Workflow	Broken agentic loop - requires human intervention

---

Recommended Fixes

Fix #1: Extract and Store Primary Topic

Where: When first user message is received in chat/planning mode

// In sendMessage or chat handler
if (messages.length === 0 || !primaryTopicRef.current) {
    // First message - extract and store primary topic
    primaryTopicRef.current = input;
    // Also save to post meta for persistence
}

Backend:

// In update_post_memory
$memory['primary_topic'] = $data['primary_topic'] ?? $memory['primary_topic'] ?? '';

Fix #2: Pass Primary Topic to Outline Generation

Where: assets/js/sidebar.js:4534

body: JSON.stringify({
    topic: primaryTopicRef.current || extractTopicFromFirstMessage(messages),
    // NOT: topic: "Create an outline based on our discussion"
})

Fix #3: Increase Chat History Limit for Outline Generation

Where: assets/js/sidebar.js:4543

// For outline generation, send MORE context
chatHistory: messages.filter(m => m.role === 'user' || m.role === 'assistant').slice(-20),
// Or better: send ALL messages for outline generation (it's a critical operation)

Fix #4: Add Topic Emphasis in System Prompt

Where: includes/class-gutenberg-sidebar.php:1723

$system_prompt = "...
CRITICAL: The PRIMARY TOPIC for this article is: {$primary_topic}
Recent refinements in the conversation are meant to REFINE this topic, not REPLACE it.
...";

Fix #5: Use Focus Keyword as Topic Anchor

Where: If user has set a focus keyword in config, prioritize it

$effective_topic = !empty($post_config['focus_keyword']) 
    ? $post_config['focus_keyword'] 
    : $topic;

---

Priority Order for Implementation

1. HIGH: Fix #2 - Pass actual topic (not generic phrase) - Quick win
2. HIGH: Fix #1 - Extract and store primary topic - Core fix
3. MEDIUM: Fix #5 - Use focus keyword as anchor - Already available
4. MEDIUM: Fix #4 - Add topic emphasis in prompt - Reinforcement
5. LOW: Fix #3 - Increase chat history limit - Already configurable in settings

---

Verification Checklist

After implementing fixes, test with this scenario:

1. Start new post
2. Enter detailed topic: "Switch career usia 30+ dengan AI dan web design"
3. Have 3-4 back-and-forth refinements (add vibe coding, focus on opinions, etc.)
4. Click "Create Outline Now"
5. VERIFY: Outline title should reference "Switch Career Usia 30+" not just "AI Web Design"
6. VERIFY: Sections should cover the FULL topic, not just recent refinements

---

Files to Modify

File	Changes
assets/js/sidebar.js	Lines 4534, 4543 - Pass extracted topic, increase history
includes/class-gutenberg-sidebar.php	Lines 1723-1765 - Add primary topic handling
includes/class-gutenberg-sidebar.php	Lines 4735-4748 - Store primary_topic in memory

---

Conclusion

The agent is "not agentic" because it doesn't remember intent - it only reacts to the most recent context. A true agentic system should:

1. Extract the user's primary intent from their first message
2. Store this intent persistently
3. Reference this intent when making decisions
4. Distinguish between refinements and new topics

The current system treats every message equally, causing recency bias to dominate and losing the user's original intent.