feat: consolidate docs, backend/session infra, and settings updates

2026-05-28 00:58:20 +07:00
parent 2424acf726
commit 44e06eed88
102 changed files with 35423 additions and 11181 deletions
--- a/docs/architecture/CORE_CHAT_WORKFLOW_GAPS.md
+++ b/docs/architecture/CORE_CHAT_WORKFLOW_GAPS.md
@@ -0,0 +1,390 @@
+# Core Chat Workflow - Gap Analysis
+
+**Document:** [docs/architecture/CORE_CHAT_WORKFLOW_SPEC.md](docs/architecture/CORE_CHAT_WORKFLOW_SPEC.md)
+**Analysis Date:** 2026-05-17
+**Status:** 🔴 ANALYSIS COMPLETE
+
+---
+
+## Executive Summary
+
+After comparing the **Core Chat Workflow Spec** against current implementation, here's what's found:
+
+| Category | Spec Items | Implemented | Gap |
+|----------|-----------|------------|-----|
+| Document States | 6 scenarios | 3 | 3 missing |
+| Mode System | 5 modes + transitions | 3 modes | 2 missing |
+| Context Management | 3 layers + optimization | 2 layers | 1 missing |
+| Intent Detection | 6 intents + actions | 4 intents | 2 missing |
+| Outline System | Full structure + refinement | Basic | Incomplete |
+| Writing Pipeline | 8 components | 5 | 3 missing |
+| Refinement | 5 types + multi-pass | 2 types | 3 missing |
+| SEO Handler | 6 components | 3 | 3 missing |
+| GEO Handler | Scoring + suggestions | None | Missing |
+| Markdown Renderer | 3 preview modes | 1 mode | 2 missing |
+
+---
+
+## Detailed Gap Analysis
+
+### 1. Document State Detection
+
+| Spec Scenario | Implementation | Status |
+|--------------|---------------|--------|
+| New post (no content, no plan) | Onboarding + chat | ✅ Done |
+| New post (from quick draft) | Shows if plan exists | ⚠️ Partial |
+| Edit existing (has content only) | Enables refinement | ⚠️ Partial |
+| Edit existing (has plan + content) | Full restore | ⚠️ Partial |
+| Re-open during writing | resume flag | ❌ Not implemented |
+| Re-open after completion | Shows completion | ❌ Not implemented |
+
+**Gap Details:**
+- ❌ Writing state not persisted to post_meta for resume
+- ❌ No `in_progress` flag detection on load
+- ❌ No completion status check and SEO prompt
+
+---
+
+### 2. Mode System
+
+| Spec Mode | Current Implementation | Status |
+|-----------|----------------------|--------|
+| `chat` | ✅ Full implementation | Done |
+| `planning` | ✅ Full implementation | Done |
+| `writing` | ✅ Full implementation | Done |
+| `refinement` | ⚠️ Basic block refinement | Partial |
+| `seo` | ❌ No dedicated SEO mode | Missing |
+
+**Current Modes in sidebar.js:**
+```javascript
+const validModes = ['chat', 'planning', 'writing', 'writing_section'];
+// Missing: refinement, seo
+```
+
+**Gap Details:**
+- ❌ No dedicated `seo` mode - SEO is scattered in UI, not a mode
+- ❌ No `refinement` mode - only block refinement exists
+- ❌ No mode history stack (for undo)
+- ❌ Mode transitions not enforced (user can go anywhere)
+
+---
+
+### 3. Context Management
+
+| Spec Component | Implementation | Status |
+|----------------|---------------|--------|
+| Chat History Context | ✅ Implemented | Done |
+| Plan Context | ✅ Implemented | Done |
+| Focus Keyword Context | ✅ Implemented | Done |
+| Context Optimization (Summarization) | ✅ `/summarize-context` exists | Done |
+| Intent Detection | ✅ `/detect-intent` exists | Done |
+| Context Indicator (UI) | ❌ Not visible | Missing |
+
+**Gap Details:**
+- ❌ No context indicator showing message/token count
+- ❌ No visual feedback when context is optimized
+- ❌ Context optimization not triggered automatically
+
+---
+
+### 4. Intent Detection
+
+| Spec Intent | Current | Status |
+|-------------|---------|--------|
+| `create_outline` | ✅ Working | Done |
+| `write_article` | ⚠️ Partial | Partial |
+| `refine_content` | ⚠️ Partial | Partial |
+| `add_section` | ❌ Not detected | Missing |
+| `clarify` | ❌ Not detected | Missing |
+| `continue_chat` | ✅ Fallback | Done |
+
+**Backend Handler (line 5627):**
+```php
+public function handle_detect_intent( $request ) {
+    // Returns intent from AI
+    // Valid intents: create_outline, start_writing, refine_content, continue_chat, clarify
+}
+```
+
+**Gap Details:**
+- ❌ `add_section` not in valid intents
+- ❌ `clarify` not handled
+- ❌ Contextual action buttons not consistently shown
+
+---
+
+### 5. Outline System
+
+| Spec Feature | Implementation | Status |
+|--------------|---------------|--------|
+| Outline data structure | ✅ Basic | Done |
+| Section metadata | ⚠️ Partial | Partial |
+| Version tracking | ❌ Not implemented | Missing |
+| Status tracking | ⚠️ Basic | Partial |
+| Drag-to-reorder | ❌ Not implemented | Missing |
+| Inline editing | ❌ Not implemented | Missing |
+
+**Current Outline Structure (simplified):**
+```javascript
+// In sidebar.js - basic structure
+{
+  sections: [
+    { id, heading, type, description, key_points, status }
+  ]
+}
+```
+
+**Spec Outline Structure (more complete):**
+```javascript
+{
+  metadata: { id, title, focus_keyword, created_at, updated_at, version, status },
+  sections: [
+    {
+      id, index, heading, type, description,
+      key_points, target_word_count, actual_word_count,
+      status, content, refinement_notes
+    }
+  ],
+  seo_notes: { primary_keyword, secondary_keywords, ... }
+}
+```
+
+**Gap Details:**
+- ❌ No version tracking on outline changes
+- ❌ No `actual_word_count` tracking
+- ❌ No `refinement_notes` per section
+- ❌ No `seo_notes` in outline structure
+- ❌ No interactive reordering
+
+---
+
+### 6. Writing Pipeline
+
+| Spec Component | Implementation | Status |
+|----------------|---------------|--------|
+| Section writing loop | ✅ Implemented | Done |
+| Writing state machine | ⚠️ Basic | Partial |
+| Pause/Resume | ⚠️ Basic | Partial |
+| Abort handling | ⚠️ Basic | Partial |
+| Block-level writing | ❌ Not implemented | Missing |
+| Writing progress persistence | ❌ Not to post_meta | Missing |
+
+**Current State:**
+- Writing state exists in React state (`agentMode === 'writing'`)
+- `resume` parameter exists for regenerating
+- `sectionInsertIndexRef` tracks where to insert
+- No full state machine (IDLE → WRITING → PAUSED → COMPLETED)
+
+**Gap Details:**
+- ❌ Writing state NOT saved to post_meta (lost on refresh)
+- ❌ No `current_section_index` persistence
+- ❌ No `sections_written[]` tracking
+- ❌ No resume from exact point on page reload
+
+---
+
+### 7. Refinement System
+
+| Refinement Type | Implementation | Status |
+|-----------------|---------------|--------|
+| Block Refinement | ✅ Full implementation | Done |
+| Section Refinement | ❌ Not implemented | Missing |
+| Article Refinement | ❌ Not implemented | Missing |
+| SEO Refinement | ❌ Not implemented | Missing |
+| Style Refinement | ❌ Not implemented | Missing |
+| Multi-Pass Refinement | ❌ Not implemented | Missing |
+
+**Current Implementation:**
+- Block refinement via `/refine-block` endpoint (line 2412)
+- Uses `@mention` syntax to select blocks
+- Context-aware (includes plan + chat history)
+- 3-pass approach mentioned in DISTRIBUTION_STRATEGY but not implemented
+
+**Gap Details:**
+- ❌ No article-wide refinement
+- ❌ No multi-pass (clarity → SEO → quality)
+- ❌ No diff display for user approval
+- ❌ No selective acceptance of changes
+
+---
+
+### 8. SEO Handler System
+
+| SEO Component | Implementation | Status |
+|---------------|---------------|--------|
+| Meta Title Generation | ⚠️ Via chat command | Partial |
+| Meta Description Generation | ⚠️ Via chat command | Partial |
+| Focus Keyword Integration | ✅ Done | Done |
+| Keyword Density Analyzer | ❌ Not implemented | Missing |
+| Content Length Checker | ❌ Not implemented | Missing |
+| Heading Structure Checker | ❌ Not implemented | Missing |
+| FAQ Generation | ❌ Not implemented | Missing |
+| Schema Markup | ❌ Not implemented | Missing |
+
+**Current SEO Implementation:**
+- Focus keyword stored in postConfig
+- Used in context for generation
+- `seo_focus_keyword` field in settings
+- No dedicated SEO analysis or suggestions
+
+**Gap Details:**
+- ❌ No SEO audit score
+- ❌ No SEO preview (Google snippet)
+- ❌ No FAQ generation with schema
+- ❌ No FAQ schema markup
+- ❌ No Article schema injection
+- ❌ No breadcrumb schema
+
+---
+
+### 9. GEO (Generative Engine Optimization) Handler
+
+| GEO Component | Implementation | Status |
+|---------------|---------------|--------|
+| GEO Score Calculation | ❌ Not implemented | Missing |
+| Directness Check | ❌ Not implemented | Missing |
+| Structure Check | ❌ Not implemented | Missing |
+| Authority Check | ❌ Not implemented | Missing |
+| Clarity Check | ❌ Not implemented | Missing |
+| GEO Improvement Suggestions | ❌ Not implemented | Missing |
+
+**Note:** GEO is a new concept (2024+) for AI-generated search results (Google SGE, Bing Chat).
+
+**Gap Details:**
+- ❌ Completely missing
+- Would need scoring system 0-100
+- Target 80+ for AI Overview eligibility
+- Suggestions for improvement
+
+---
+
+### 10. Markdown Rendering System
+
+| Renderer Feature | Implementation | Status |
+|------------------|---------------|--------|
+| Markdown to HTML | ✅ Implemented (markdown-it) | Done |
+| Syntax highlighting | ❌ Not implemented | Missing |
+| Collapsible headings | ❌ Not implemented | Missing |
+| Quick copy button | ❌ Not implemented | Missing |
+| WYSIWYG Preview mode | ❌ Not implemented | Missing |
+| Split View mode | ❌ Not implemented | Missing |
+| Code block language detection | ❌ Not implemented | Missing |
+
+**Current Implementation:**
+```javascript
+// sidebar.js line 4975
+const markdownToHtml = (markdown) => {
+    // Uses markdown-it library
+    // Uses DOMPurify for sanitization
+}
+```
+
+**Gap Details:**
+- ❌ No toggle between Preview/Markdown/Split modes
+- ❌ No code syntax highlighting (highlight.js)
+- ❌ No copy-to-clipboard for sections
+- ❌ No @mention highlighting in preview
+
+---
+
+## Summary: Missing Features by Priority
+
+### 🔴 HIGH PRIORITY (Core Functionality)
+
+1. **Writing State Persistence** ✅ DONE
+   - Save to post_meta: current_section_index, sections_written[], content
+   - Enable seamless resume after page reload
+
+2. **SEO Mode (Dedicated)** ✅ DONE
+   - Create SEO tab/mode in sidebar
+   - Meta title/description generation
+   - SEO preview (Google snippet)
+
+3. **SEO Handler Components** ✅ DONE
+   - FAQ generation (via SEO tab)
+   - Schema markup (Article, FAQ, Breadcrumb) - partial
+   - Keyword density checker
+
+### 🟡 MEDIUM PRIORITY (Enhanced UX)
+
+4. **Outline Enhancement** ✅ DONE
+   - Version tracking ✅
+   - Drag-to-reorder sections ✅
+   - Inline editing ✅
+   - `actual_word_count` tracking ✅
+
+5. **Refinement System Expansion** ✅ DONE
+   - Article-wide refinement ✅
+   - Multi-pass refinement (3 stages) ✅
+   - Refinement action buttons ✅
+
+6. **Context Optimization UI** ✅ DONE
+   - Context indicator (message count, token estimate)
+   - Visual feedback when context is optimized
+
+### 🟢 LOW PRIORITY (Polish)
+
+7. **Markdown Renderer Enhancement** ✅ DONE (partial)
+   - Copy button for sections ✅
+
+8. **GEO Handler** ✅ DONE
+   - GEO scoring system ✅
+   - 5-checks scoring (Directness, Structure, Authority, Clarity, Completeness) ✅
+   - AI Overview eligibility indicator ✅
+   - Improvement suggestions ✅
+
+---
+
+## Implementation Roadmap
+
+```
+Phase 1: Core Foundation (Critical) ✅ COMPLETE
+├── 1.1 Writing State Persistence to post_meta ✅ DONE
+├── 1.2 SEO Mode (Dedicated tab) ✅ DONE
+├── 1.3 Meta Title/Description Generator ✅ DONE
+└── 1.4 SEO Preview (Google snippet) ✅ DONE
+
+Phase 2: SEO Handler ✅ COMPLETE
+├── 2.1 FAQ Generation ✅ DONE (via SEO tab)
+├── 2.2 Schema Markup (Article, FAQ) ✅ PARTIAL
+├── 2.3 Keyword Density Checker ✅ DONE
+└── 2.4 SEO Audit Score ✅ DONE
+
+Phase 3: Outline Enhancement ✅ COMPLETE
+├── 3.1 Version Tracking ✅ DONE
+├── 3.2 Drag-to-Reorder ✅ DONE
+├── 3.3 Inline Editing ✅ DONE
+└── 3.4 Word Count Tracking ✅ DONE
+
+Phase 4: Refinement Expansion ✅ COMPLETE
+├── 4.1 Article-Wide Refinement ✅ DONE
+├── 4.2 Multi-Pass Refinement ✅ DONE
+└── 4.3 Refinement Actions UI ✅ DONE
+
+Phase 5: Polish ✅ COMPLETE
+├── 5.1 Context Indicator UI ✅ DONE
+├── 5.2 Copy Button ✅ DONE
+└── 5.3 GEO Handler ✅ DONE
+
+Phase 6: WP 7.0 AI Integration ✅ COMPLETE
+├── 6.1 WP AI Client Detection ✅ DONE
+├── 6.2 Backward-Compatible Wrapper ✅ DONE
+├── 6.3 REST Endpoints for Title/Excerpt ✅ DONE
+└── 6.4 Settings Integration ✅ DONE
+```
+
+---
+
+## Files to Modify
+
+| File | Changes Needed |
+|------|---------------|
+| `assets/js/sidebar.js` | State persistence, SEO mode, context indicator |
+| `includes/class-gutenberg-sidebar.php` | SEO endpoints, schema generation |
+| `assets/css/sidebar.css` | New UI components |
+| `views/settings/tab-general.php` | SEO settings |
+
+---
+
+**Analysis Completed:** 2026-05-17
+**Next Action:** Choose which gap to address first (recommended: 1.1 Writing State Persistence)