wp-agentic-writer/CONTEXT_GAP_DIAGNOSTIC_REPORT.md

# Context Gap Diagnostic Report

**Date:** January 30, 2026  
**Issue:** Context lost during outline generation - Agent doesn't maintain conversation continuity  
**Severity:** High - Core agentic behavior broken

---

## Executive Summary

The agent loses context when generating outlines because:
1. **Generic topic passed** instead of extracting actual topic from conversation
2. **Chat history truncated** to last 10 messages (first message with core topic lost)
3. **No topic extraction mechanism** - system relies on recency, not importance
4. **Recency bias** - LLM sees recent refinements more than original intent

---

## Conversation Flow Analysis

### User's Conversation Timeline

| Step | User Action | Agent Response | Context Status |
|------|-------------|----------------|----------------|
| 1 | Rich topic: "switch career usia 30+, AI, web design, vibe coding..." | Comprehensive response | FULL CONTEXT |
| 2 | "tambahkan vibe coding" | Added vibe coding section | FULL CONTEXT |
| 3 | "fokus opini, web design, tanpa coding" | Refined to web design focus | FULL CONTEXT |
| 4 | Click "Create Outline Now" | Generated outline about "AI Web Design" | CONTEXT LOST |
| 5 | User manually sets focus keyword, asks to redo | Regenerated with correct focus | RECOVERED (manually) |

### Where Context Was Lost

**Step 4: "Create Outline Now" button click**

The outline focused on "AI-Powered Web Design" instead of the broader "Switch Career Usia 30+" topic that was the user's original intent.

---

## Root Cause Analysis

### Defect #1: Generic Topic Parameter

**Location:** `assets/js/sidebar.js:4534`

```javascript
body: JSON.stringify({
    topic: outlineMessage,  // "Create an outline based on our discussion"
    // ...
})
```

**Problem:** The `topic` variable is set to the literal string `"Create an outline based on our discussion"` instead of extracting the actual topic from the first user message.

**Impact:** The LLM receives a generic topic and must infer intent from chat history alone.

---

### Defect #2: Chat History Truncation (.slice(-10))

**Location:** `assets/js/sidebar.js:4543`

```javascript
chatHistory: messages.filter(m => m.role === 'user' || m.role === 'assistant').slice(-10),
```

**Problem:** Only the **last 10 messages** are sent to the backend. If the conversation has more than 10 exchanges, the **first user message (which contains the core topic)** is lost.

**Your Conversation Analysis:**
- Message 1: User's detailed topic request (CRITICAL - contains "switch career usia 30+")
- Message 2: Agent's comprehensive response
- Message 3: User adds "vibe coding"
- Message 4: Agent responds about vibe coding
- Message 5: User refines to "web design focus"
- Message 6: Agent responds about web design
- Message 7: User clicks "Create Outline Now" (adds another message)

With `.slice(-10)`, the first message **might still be included** in this case, but the truncation creates fragility. The real issue is combined with Defect #1.

---

### Defect #3: No Topic Extraction Mechanism

**Location:** `includes/class-gutenberg-sidebar.php:1765`

```php
'content' => "Topic: {$topic}\n\nContext: {$context}{$chat_history_context}..."
```

**Problem:** The system doesn't extract the user's **original topic/intent** from the first message. It just appends chat history as context, but the LLM prompt structure puts emphasis on `{$topic}` which is generic.

**What should happen:**
1. Extract topic from first user message
2. Store as "primary topic" in post memory
3. Use primary topic in outline generation, not generic phrase

---

### Defect #4: Recency Bias in LLM Processing

**Problem:** When the LLM sees the chat history, the **most recent messages** (about web design) appear at the end and have more weight than earlier messages about the broader topic.

**Chat History Seen by LLM:**
```
User: [switch career usia 30+ topic...] ← EARLY, less weight
Assistant: [comprehensive response...]
User: [add vibe coding...]
Assistant: [vibe coding response...]
User: [focus on web design...] ← RECENT, more weight
Assistant: [web design response...] ← MOST RECENT, highest weight
User: Create an outline based on our discussion
```

The LLM naturally focuses on the most recent topic (web design) rather than the original broader topic.

---

### Defect #5: Memory System Doesn't Store Primary Topic

**Location:** `includes/class-gutenberg-sidebar.php:4735-4748`

```php
private function update_post_memory( $post_id, $data ) {
    // Only stores: summary, last_prompt, last_intent
    // Does NOT store: primary_topic, original_intent, focus_keyword
}
```

**Problem:** The memory system stores `last_prompt` but not `primary_topic`. When generating an outline, there's no reference to what the user originally wanted.

---

## Impact Analysis

| Aspect | Impact |
|--------|--------|
| User Experience | Frustrating - user must manually correct agent's misunderstanding |
| Cost | Wasted API calls on incorrect outline generation |
| Trust | User loses confidence in agent's ability to understand context |
| Workflow | Broken agentic loop - requires human intervention |

---

## Recommended Fixes

### Fix #1: Extract and Store Primary Topic

**Where:** When first user message is received in chat/planning mode

```javascript
// In sendMessage or chat handler
if (messages.length === 0 || !primaryTopicRef.current) {
    // First message - extract and store primary topic
    primaryTopicRef.current = input;
    // Also save to post meta for persistence
}
```

**Backend:**
```php
// In update_post_memory
$memory['primary_topic'] = $data['primary_topic'] ?? $memory['primary_topic'] ?? '';
```

### Fix #2: Pass Primary Topic to Outline Generation

**Where:** `assets/js/sidebar.js:4534`

```javascript
body: JSON.stringify({
    topic: primaryTopicRef.current || extractTopicFromFirstMessage(messages),
    // NOT: topic: "Create an outline based on our discussion"
})
```

### Fix #3: Increase Chat History Limit for Outline Generation

**Where:** `assets/js/sidebar.js:4543`

```javascript
// For outline generation, send MORE context
chatHistory: messages.filter(m => m.role === 'user' || m.role === 'assistant').slice(-20),
// Or better: send ALL messages for outline generation (it's a critical operation)
```

### Fix #4: Add Topic Emphasis in System Prompt

**Where:** `includes/class-gutenberg-sidebar.php:1723`

```php
$system_prompt = "...
CRITICAL: The PRIMARY TOPIC for this article is: {$primary_topic}
Recent refinements in the conversation are meant to REFINE this topic, not REPLACE it.
...";
```

### Fix #5: Use Focus Keyword as Topic Anchor

**Where:** If user has set a focus keyword in config, prioritize it

```php
$effective_topic = !empty($post_config['focus_keyword']) 
    ? $post_config['focus_keyword'] 
    : $topic;
```

---

## Priority Order for Implementation

1. **HIGH: Fix #2** - Pass actual topic (not generic phrase) - Quick win
2. **HIGH: Fix #1** - Extract and store primary topic - Core fix
3. **MEDIUM: Fix #5** - Use focus keyword as anchor - Already available
4. **MEDIUM: Fix #4** - Add topic emphasis in prompt - Reinforcement
5. **LOW: Fix #3** - Increase chat history limit - Already configurable in settings

---

## Verification Checklist

After implementing fixes, test with this scenario:

1. Start new post
2. Enter detailed topic: "Switch career usia 30+ dengan AI dan web design"
3. Have 3-4 back-and-forth refinements (add vibe coding, focus on opinions, etc.)
4. Click "Create Outline Now"
5. **VERIFY:** Outline title should reference "Switch Career Usia 30+" not just "AI Web Design"
6. **VERIFY:** Sections should cover the FULL topic, not just recent refinements

---

## Files to Modify

| File | Changes |
|------|---------|
| `assets/js/sidebar.js` | Lines 4534, 4543 - Pass extracted topic, increase history |
| `includes/class-gutenberg-sidebar.php` | Lines 1723-1765 - Add primary topic handling |
| `includes/class-gutenberg-sidebar.php` | Lines 4735-4748 - Store primary_topic in memory |

---

## Conclusion

The agent is "not agentic" because it **doesn't remember intent** - it only reacts to the most recent context. A true agentic system should:

1. **Extract** the user's primary intent from their first message
2. **Store** this intent persistently
3. **Reference** this intent when making decisions
4. **Distinguish** between refinements and new topics

The current system treats every message equally, causing recency bias to dominate and losing the user's original intent.