wp-agentic-writer/IMPLEMENTATION_PLAN-clarification-quiz.md

Implementation Plan: Enhanced Clarification Quiz System

Overview

Improve the clarification quiz to appear more frequently and gather comprehensive contextual information (target outcome, education level, marketing type, etc.) using predefined options users can select from.

---

Phase 1: Add Settings Configuration

File: includes/class-settings.php

Location: Add new section after existing settings (around line 200+)

New Settings to Add:

/**
 * Clarification Quiz Settings Section
 */
public function add_clarification_quiz_settings() {
    add_settings_section(
        'wp_aw_clarification_quiz',
        __( 'Clarification Quiz', 'wp-agentic-writer' ),
        array( $this, 'clarification_quiz_section_callback' ),
        'wp-agentic-writer'
    );

    // Enable/Disable Quiz
    add_settings_field(
        'enable_clarification_quiz',
        __( 'Enable Clarification Quiz', 'wp-agentic-writer' ),
        array( $this, 'render_checkbox' ),
        'wp-agentic-writer',
        'wp_aw_clarification_quiz',
        array(
            'label_for' => 'enable_clarification_quiz',
            'default'   => true,
            'description' => __( 'Automatically ask clarifying questions when context is missing.', 'wp-agentic-writer' ),
        )
    );

    // Confidence Threshold
    add_settings_field(
        'clarity_confidence_threshold',
        __( 'Confidence Threshold', 'wp-agentic-writer' ),
        array( $this, 'render_select' ),
        'wp-agentic-writer',
        'wp_aw_clarification_quiz',
        array(
            'label_for' => 'clarity_confidence_threshold',
            'default'   => '0.6',
            'options'   => array(
                '0.5' => __( 'Very Sensitive (50%) - Quiz appears frequently', 'wp-agentic-writer' ),
                '0.6' => __( 'Sensitive (60%) - Recommended', 'wp-agentic-writer' ),
                '0.7' => __( 'Balanced (70%)', 'wp-agentic-writer' ),
                '0.8' => __( 'Strict (80%) - Current default', 'wp-agentic-writer' ),
                '0.9' => __( 'Very Strict (90%) - Quiz rarely appears', 'wp-agentic-writer' ),
            ),
            'description' => __( 'Lower threshold = quiz appears more often. Higher threshold = only very unclear requests trigger quiz.', 'wp-agentic-writer' ),
        )
    );

    // Required Context Categories
    add_settings_field(
        'required_context_categories',
        __( 'Required Context', 'wp-agentic-writer' ),
        array( $this, 'render_multiselect' ),
        'wp-agentic-writer',
        'wp_aw_clarification_quiz',
        array(
            'label_for' => 'required_context_categories',
            'default'   => array( 'target_outcome', 'target_audience', 'tone', 'content_depth', 'expertise_level', 'content_type', 'pov' ),
            'options'   => array(
                'target_outcome'  => __( 'Target Outcome (education/marketing/sales)', 'wp-agentic-writer' ),
                'target_audience' => __( 'Target Audience (who reads this)', 'wp-agentic-writer' ),
                'tone'            => __( 'Tone of Voice (formal/casual/technical)', 'wp-agentic-writer' ),
                'content_depth'   => __( 'Content Depth (overview/guide/analysis)', 'wp-agentic-writer' ),
                'expertise_level' => __( 'Expertise Level (beginner/intermediate/advanced)', 'wp-agentic-writer' ),
                'content_type'    => __( 'Content Type (tutorial/opinion/how-to)', 'wp-agentic-writer' ),
                'pov'             => __( 'Point of View (first/third person)', 'wp-agentic-writer' ),
            ),
            'description' => __( 'Select which context categories must be clear before writing. Uncheck categories you don\'t need.', 'wp-agentic-writer' ),
        )
    );

    register_setting( 'wp-agentic-writer', 'enable_clarification_quiz' );
    register_setting( 'wp-agentic-writer', 'clarity_confidence_threshold' );
    register_setting( 'wp-agentic-writer', 'required_context_categories' );
}

Helper Methods to Add:

/**
 * Render multiselect field
 */
public function render_multiselect( $args ) {
    $name = $args['label_for'];
    $value = get_option( $name, $args['default'] );
    if ( ! is_array( $value ) ) {
        $value = $args['default'];
    }

    echo '<select multiple name="' . esc_attr( $name ) . '[]" id="' . esc_attr( $name ) . '" class="regular-text">';
    foreach ( $args['options'] as $key => $label ) {
        $selected = in_array( $key, $value ) ? 'selected' : '';
        echo '<option value="' . esc_attr( $key ) . '" ' . $selected . '>' . esc_html( $label ) . '</option>';
    }
    echo '</select>';

    if ( isset( $args['description'] ) ) {
        echo '<p class="description">' . wp_kses_post( $args['description'] ) . '</p>';
    }
}

public function clarification_quiz_section_callback() {
    echo '<p>' . __( 'Configure when and how the clarification quiz appears to gather context for better article generation.', 'wp-agentic-writer' ) . '</p>';
}

Hook Integration: Add to __construct() or settings initialization:

add_action( 'admin_init', array( $this, 'add_clarification_quiz_settings' ), 20 );

---

Phase 2: Update Clarity Check System Prompt

File: includes/class-gutenberg-sidebar.php

Location: Lines 1195-1231 (check_clarity method)

Replace the current system prompt with:

$required_categories = get_option( 'required_context_categories', array(
    'target_outcome',
    'target_audience',
    'tone',
    'content_depth',
    'expertise_level',
    'content_type',
    'pov'
) );

$threshold = get_option( 'clarity_confidence_threshold', '0.6' );
$enabled = get_option( 'enable_clarification_quiz', true );

// If quiz is disabled, always return clear
if ( ! $enabled ) {
    return new WP_REST_Response(
        array(
            'result' => array(
                'is_clear' => true,
                'confidence' => 1.0,
                'questions' => array()
            ),
        ),
        200
    );
}

$system_prompt = "You are an expert editor who determines if an article request has sufficient context to write effectively.

Evaluate the user's request and determine which context categories are clear:

CATEGORIES TO EVALUATE:
1. target_outcome - What should this content achieve? (education/marketing/sales/entertainment/brand_awareness)
2. target_audience - Who is reading this? (demographics, role, knowledge level)
3. tone - How should we sound? (formal/casual/technical/friendly/professional/conversational)
4. content_depth - How comprehensive? (quick_overview/standard_guide/detailed_analysis/comprehensive)
5. expertise_level - Reader's knowledge? (beginner/intermediate/advanced/expert)
6. content_type - What format? (tutorial/how_to/opinion/comparison/listicle/case_study/news_analysis)
7. pov - Whose perspective? (first_person/third_person/expert_voice/neutral)

For each MISSING category, generate a clarifying question using PREDEFINED OPTIONS.
Use 'single_choice' or 'multiple_choice' types - NEVER 'open_text'.

QUESTION STRUCTURE:
{
  'id': 'q1',
  'category': 'target_outcome',
  'question': 'What is the primary goal of this content?',
  'type': 'single_choice',
  'options': [
    { 'value': 'Education - Teach something new', 'default': true },
    { 'value': 'Marketing - Promote a product/service', 'default': false },
    { 'value': 'Sales - Drive conversions/signups', 'default': false },
    { 'value': 'Entertainment - Engage and entertain', 'default': false },
    { 'value': 'Brand Awareness - Build authority/trust', 'default': false }
  ]
}

CONFIDENCE CALCULATION:
- Start at 100% (1.0)
- Subtract 15% for each missing required category
- If confidence < {$threshold}, generate questions for ALL missing categories

Return ONLY valid JSON with this structure:
{
  'is_clear': true/false,
  'confidence': 0.0-1.0,
  'missing_categories': ['category1', 'category2'],
  'questions': [ ... ]
}

No markdown, no explanation - just JSON.";

$messages = array(
    array(
        'role'    => 'system',
        'content' => $system_prompt,
    ),
    array(
        'role'    => 'user',
        'content' => "Topic: {$topic}\n\nRequired Categories: " . implode( ', ', $required_categories ) . "\n\nEvaluate this request and determine which context is missing.",
    ),
);

---

Phase 3: Improve Fallback Behavior

File: includes/class-gutenberg-sidebar.php

Location: Around lines 1603-1615

Add new helper method:

/**
 * Get default clarification questions when AI fails
 *
 * @since 0.1.0
 * @param string $topic User's topic.
 * @return array Clarification result with default questions.
 */
private function get_default_clarification_questions( $topic ) {
    $required_categories = get_option( 'required_context_categories', array(
        'target_outcome',
        'target_audience',
        'tone',
        'content_depth',
        'expertise_level',
        'content_type',
        'pov'
    ) );

    $questions = array();
    $question_id = 1;

    $question_templates = array(
        'target_outcome' => array(
            'category' => 'target_outcome',
            'question' => 'What is the primary goal of this content?',
            'type' => 'single_choice',
            'options' => array(
                array( 'value' => 'Education - Teach something new', 'default' => true ),
                array( 'value' => 'Marketing - Promote a product/service', 'default' => false ),
                array( 'value' => 'Sales - Drive conversions', 'default' => false ),
                array( 'value' => 'Entertainment - Engage readers', 'default' => false ),
                array( 'value' => 'Brand Awareness - Build authority', 'default' => false ),
            )
        ),
        'target_audience' => array(
            'category' => 'target_audience',
            'question' => 'Who is the primary audience for this content?',
            'type' => 'single_choice',
            'options' => array(
                array( 'value' => 'General public / Beginners', 'default' => true ),
                array( 'value' => 'Professionals in the field', 'default' => false ),
                array( 'value' => 'Potential customers', 'default' => false ),
                array( 'value' => 'Existing customers/users', 'default' => false ),
                array( 'value' => 'Industry peers / Experts', 'default' => false ),
            )
        ),
        'tone' => array(
            'category' => 'tone',
            'question' => 'What tone should this content have?',
            'type' => 'single_choice',
            'options' => array(
                array( 'value' => 'Professional & Authoritative', 'default' => true ),
                array( 'value' => 'Friendly & Conversational', 'default' => false ),
                array( 'value' => 'Technical & Detailed', 'default' => false ),
                array( 'value' => 'Casual & Entertaining', 'default' => false ),
                array( 'value' => 'Formal & Academic', 'default' => false ),
            )
        ),
        'content_depth' => array(
            'category' => 'content_depth',
            'question' => 'How comprehensive should this content be?',
            'type' => 'single_choice',
            'options' => array(
                array( 'value' => 'Quick overview (500-800 words)', 'default' => false ),
                array( 'value' => 'Standard guide (800-1500 words)', 'default' => true ),
                array( 'value' => 'Detailed analysis (1500-2500 words)', 'default' => false ),
                array( 'value' => 'Comprehensive deep-dive (2500+ words)', 'default' => false ),
            )
        ),
        'expertise_level' => array(
            'category' => 'expertise_level',
            'question' => 'What is the target audience\'s expertise level?',
            'type' => 'single_choice',
            'options' => array(
                array( 'value' => 'Beginner - No prior knowledge', 'default' => true ),
                array( 'value' => 'Intermediate - Basic understanding', 'default' => false ),
                array( 'value' => 'Advanced - Deep technical knowledge', 'default' => false ),
                array( 'value' => 'Expert - Industry professional', 'default' => false ),
            )
        ),
        'content_type' => array(
            'category' => 'content_type',
            'question' => 'What type of content works best for this topic?',
            'type' => 'single_choice',
            'options' => array(
                array( 'value' => 'Tutorial / How-to guide', 'default' => true ),
                array( 'value' => 'Opinion / Commentary', 'default' => false ),
                array( 'value' => 'Comparison / Review', 'default' => false ),
                array( 'value' => 'Listicle / Tips', 'default' => false ),
                array( 'value' => 'Case study', 'default' => false ),
                array( 'value' => 'News analysis', 'default' => false ),
            )
        ),
        'pov' => array(
            'category' => 'pov',
            'question' => 'From what perspective should this be written?',
            'type' => 'single_choice',
            'options' => array(
                array( 'value' => 'Third person (objective, "it", "they")', 'default' => true ),
                array( 'value' => 'First person (personal, "I", "my")', 'default' => false ),
                array( 'value' => 'Expert voice (authoritative, experienced)', 'default' => false ),
                array( 'value' => 'Neutral / Unbiased', 'default' => false ),
            )
        ),
    );

    foreach ( $required_categories as $category ) {
        if ( isset( $question_templates[ $category ] ) ) {
            $q = $question_templates[ $category ];
            $q['id'] = 'q' . $question_id++;
            $questions[] = $q;
        }
    }

    return array(
        'is_clear' => false,
        'confidence' => 0.0,
        'missing_categories' => $required_categories,
        'questions' => $questions
    );
}

Update error handling (lines 1603-1615):

if ( is_wp_error( $response ) ) {
    // Log error
    error_log( 'WP Agentic Writer: Clarity check API error - ' . $response->get_error_message() );
    // Use default questions instead of skipping
    return $this->get_default_clarification_questions( $topic );
}

if ( null === $result ) {
    // Log parse error
    error_log( 'WP Agentic Writer: Failed to parse clarity check JSON' );
    // Use default questions instead of skipping
    return $this->get_default_clarification_questions( $topic );
}

---

Phase 4: Update Frontend Quiz Display

File: assets/js/sidebar.js

Enhancement 1: Add category labels to questions

Find the question rendering code and add category display:

// Inside renderClarificationQuestion function
const categoryLabels = {
    'target_outcome': '🎯 Target Outcome',
    'target_audience': '👥 Target Audience',
    'tone': '🎨 Tone of Voice',
    'content_depth': '📏 Content Depth',
    'expertise_level': '📊 Expertise Level',
    'content_type': '📝 Content Type',
    'pov': '👁️ Point of View'
};

// Add category badge above question
if (question.category && categoryLabels[question.category]) {
    html += '<div class="clarification-category-badge">';
    html += categoryLabels[question.category];
    html += '</div>';
}

Enhancement 2: Show which categories are clear

Update progress display to show collected vs missing context:

// Update progress display
function updateClarificationProgress(total, current, clearCategories = []) {
    const progressEl = document.querySelector('.clarification-progress');
    if (!progressEl) return;

    let html = `<div class="progress-info">`;
    html += `<span>Question ${current} of ${total}</span>`;

    // Show what we already know
    if (clearCategories.length > 0) {
        html += `<div class="clear-context">`;
        html += `<strong>Already clear:</strong> `;
        html += clearCategories.map(cat => {
            const labels = {
                'target_outcome': 'Goal',
                'target_audience': 'Audience',
                'tone': 'Tone',
                'content_depth': 'Depth',
                'expertise_level': 'Level',
                'content_type': 'Format',
                'pov': 'Perspective'
            };
            return labels[cat] || cat;
        }).join(', ');
        html += `</div>`;
    }

    html += `</div>`;
    html += `<div class="progress-bar"><div class="progress-fill" style="width: ${(current/total)*100}%"></div></div>`;

    progressEl.innerHTML = html;
}

Enhancement 3: Add "Skip this question" option

// Add skip button to question card
html += `<button class="skip-question-btn" data-question-id="${question.id}">`;
html += `Skip (not applicable)`;
html += `</button>`;

// Handle skip
document.addEventListener('click', (e) => {
    if (e.target.classList.contains('skip-question-btn')) {
        const questionId = e.target.dataset.questionId;
        skipQuestion(questionId);
    }
});

function skipQuestion(questionId) {
    // Mark as skipped with "Not applicable" value
    clarificationAnswers[questionId] = {
        skipped: true,
        value: 'Not applicable'
    };
    nextClarificationQuestion();
}

---

Phase 5: Pass Clarification Context to Plan Generation

File: includes/class-gutenberg-sidebar.php

Location: Find generate_plan method (around line 900+)

Add clarification context integration:

// Before generating the plan, check if we have clarification answers
$clarity_context = '';
if ( ! empty( $params['clarificationAnswers'] ) && is_array( $params['clarificationAnswers'] ) ) {
    $clarity_context = "\n\n=== CONTEXT FROM CLARIFICATION QUIZ ===\n";

    // Group by category
    $grouped = array();
    foreach ( $params['clarificationAnswers'] as $answer ) {
        $category = $answer['category'] ?? 'other';
        $value = $answer['value'] ?? $answer['answer'] ?? '';
        $skipped = $answer['skipped'] ?? false;

        if ( ! $skipped && ! empty( $value ) ) {
            $grouped[ $category ] = $value;
        }
    }

    // Format for prompt
    $category_labels = array(
        'target_outcome'  => 'Primary Goal',
        'target_audience' => 'Target Audience',
        'tone'            => 'Tone of Voice',
        'content_depth'   => 'Content Depth',
        'expertise_level' => 'Expertise Level',
        'content_type'    => 'Content Type',
        'pov'             => 'Point of View',
    );

    foreach ( $grouped as $category => $value ) {
        $label = $category_labels[ $category ] ?? ucwords( str_replace( '_', ' ', $category ) );
        $clarity_context .= "- {$label}: {$value}\n";
    }

    $clarity_context .= "=== END CONTEXT ===\n";
}

// Add to planning prompt
$plan_prompt = $clarity_context . "\n" . $plan_prompt;

---

Phase 6: Testing Checklist

Manual Testing Steps:

1. Settings Page Test:

   • Go to WP Agentic Writer settings
   • Verify new "Clarification Quiz" section appears
   • Test enable/disable toggle
   • Change confidence threshold to different values
   • Select/deselect required context categories
   • Save settings and verify they persist

2. Vague Topic Test:

   • Enter very vague topic: "write about AI"
   • Verify quiz appears with all missing categories
   • Verify all questions have predefined options (no text inputs)
   • Answer all questions and verify plan reflects choices

3. Specific Topic Test:

   • Enter detailed topic with all context
   • Verify quiz doesn't appear (or only asks for truly missing info)
   • Plan generation proceeds immediately

4. Threshold Test:

   • Set threshold to 0.5 (very sensitive)
   • Enter semi-clear topic
   • Verify quiz appears
   • Set threshold to 0.9 (very strict)
   • Enter same topic
   • Verify quiz doesn't appear

5. Category Filter Test:

   • Uncheck some categories in settings
   • Enter vague topic
   • Verify quiz only asks for checked categories
   • Unchecked categories are skipped

6. API Failure Test:

   • Temporarily break API connection (invalid key)
   • Enter vague topic
   • Verify fallback questions appear
   • All categories show default predefined options

7. Skip Question Test:

   • Start quiz
   • Click "Skip" on a question
   • Verify it moves to next question
   • Verify skipped answer marked as "Not applicable"

8. Plan Integration Test:

   • Complete full quiz with specific answers
   • Generate plan
   • Verify plan reflects quiz answers (tone, audience, goal, etc.)

---

Success Criteria

✅ Settings page has Clarification Quiz section with all 3 fields ✅ Quiz triggers more frequently with 0.6 threshold (vs 0.8) ✅ All quiz questions use predefined options (no open_text type) ✅ Fallback questions appear when AI fails ✅ Clarification answers appear in generated plan ✅ Users can configure which categories are required ✅ Users can enable/disable quiz entirely ✅ Frontend shows category labels and progress ✅ Can skip individual questions

---

Rollback Plan

If issues occur:

1. Revert class-settings.php changes (remove new settings)
2. Revert system prompt to original (simple 3-criteria check)
3. Revert threshold to hardcoded 0.8
4. Revert fallback to original "assume clear" behavior

Keep Git commits organized by phase for easy partial rollback.

---

Future Enhancements (Out of Scope)

• Add custom question types via filters
• Save quiz answers per user/site for faster future generations
• A/B test different thresholds
• Analytics on which categories are most often missing
• Import/export context presets
• Multi-language support for question options