wp-agentic-writer/brave_search_integration.md

WP Agentic Writer: Brave Search API Integration Guide

Executive Summary

This document defines the complete integration of Brave Search API into WP Agentic Writer to enhance article generation with real-time web research, citations, and fact-grounding.

Key principle: During article generation, the writing agent calls Brave Search API to fetch current data, verify facts, gather citations, and ground responses in real sources. All search results are logged, cached, and attributed.

---

Table of Contents

1. Overview & Architecture
2. Brave Search API Fundamentals
3. Data Model & Database Schema
4. Flow 1: Agent Research Planning
5. Flow 2: Brave Search API Integration
6. Flow 3: Citation Management
7. Flow 4: Search Result Caching
8. Flow 5: Admin Dashboard & Analytics
9. REST API Endpoints
10. Configuration & Settings
11. Cost Optimization & Budget Management
12. Implementation Checklist

---

Overview & Architecture

Core Concept

┌─────────────────────────────────────────────────────────────────┐
│ USER: "Write article about N8n automation workflows"           │
└────────────────────┬────────────────────────────────────────────┘
                     ↓
┌─────────────────────────────────────────────────────────────────┐
│ WRITING AGENT receives request                                  │
│ - Analyzes: topic, target audience, depth needed               │
│ - Plans research: "Need current N8n features, pricing, users"  │
└────────────────────┬────────────────────────────────────────────┘
                     ↓
┌─────────────────────────────────────────────────────────────────┐
│ AGENT → Brave Search API (Multiple searches)                    │
│                                                                  │
│ Search 1: "N8n automation platform 2024"                        │
│ Search 2: "N8n pricing plans features"                          │
│ Search 3: "N8n vs Zapier Make comparison"                       │
│ Search 4: "N8n self-hosted deployment guide"                    │
│                                                                  │
│ Each search: Get 10-15 results with snippets + URLs            │
└────────────────────┬────────────────────────────────────────────┘
                     ↓
┌─────────────────────────────────────────────────────────────────┐
│ BACKEND: Store searches in DB                                   │
│                                                                  │
│ wp_agentic_searches table:                                      │
│ - post_id, search_query, results (serialized), cost             │
│ - cache_enabled (reuse for 30 days)                            │
│                                                                  │
│ wp_agentic_citations table:                                     │
│ - post_id, citation_text, url, source_title, position          │
│ - [1], [2], [3] numbered references                            │
└────────────────────┬────────────────────────────────────────────┘
                     ↓
┌─────────────────────────────────────────────────────────────────┐
│ AGENT: Generates article WITH research                          │
│                                                                  │
│ "N8n is an open-source workflow automation platform[1]          │
│  enabling teams to automate tasks[2]. In 2024, N8n added[3]...  │
│                                                                  │
│  Key features include visual workflow builder[1],               │
│  500+ integrations[2], and self-hosting options[3]."            │
│                                                                  │
│ Returns: Article blocks + citations (with sources)             │
└────────────────────┬────────────────────────────────────────────┘
                     ↓
┌─────────────────────────────────────────────────────────────────┐
│ PLUGIN: Create Gutenberg post with:                            │
│ - Article content                                               │
│ - Inline citations [1], [2], [3]                               │
│ - References section at end (with URLs)                        │
│ - Search context sidebar (optional)                            │
└────────────────────┬────────────────────────────────────────────┘
                     ↓
┌─────────────────────────────────────────────────────────────────┐
│ ADMIN DASHBOARD: "Generated Searches"                           │
│                                                                  │
│ Show:                                                            │
│ - All searches per post + results count                        │
│ - Cost per search ($0.009 per query for AI tier)               │
│ - Cache status (reusable until expiry)                         │
│ - Citation coverage (% of facts cited)                         │
└─────────────────────────────────────────────────────────────────┘

---

Brave Search API Fundamentals

API Plans & Pricing (2024)

Free Tier (for testing):

• 1 query/second
• 2,000 queries/month (~$0)
• Great for development & testing

Data for AI - Base ($5 per 1,000 requests):

• 20 queries/second
• Up to 20M queries/month
• Up to 5 snippets per result
• Extra alternate snippets for AI
• Rights to use in AI apps

Data for AI - Pro ($9 per 1,000 requests):

• 50 queries/second
• Unlimited queries/month
• Same features as Base + schema-enriched results
• Recommended for production AI apps

Key Advantages Over Alternatives

Feature	Brave Search	Google Search	Bing Search
Cost	$3-9 CPM	~$20 CPM	~$15 CPM
Independent Index	Yes (30B+ pages)	No (proprietary)	No (proprietary)
Privacy	Yes (no tracking)	No	No
AI Rights	Explicit in plan	Unclear	Unclear
Fresh Data	100M updates/day	Daily	Daily
Latency	~2-3s average	Fast	Fast

---

Data Model & Database Schema

Table: wp_agentic_searches

Stores all Brave Search API calls made during article generation.

CREATE TABLE wp_agentic_searches (
    id BIGINT AUTO_INCREMENT PRIMARY KEY,
    post_id BIGINT NOT NULL,
    
    -- Search details
    search_query VARCHAR(500) NOT NULL,       -- "N8n automation features"
    search_number INT,                        -- 1st, 2nd, 3rd search
    total_searches_for_post INT,              -- "Need 3 searches for this post"
    
    -- Brave API response
    results_count INT,                        -- How many results returned
    results_json LONGTEXT,                    -- Serialized JSON from Brave
    top_result_title VARCHAR(255),            -- First result title
    top_result_url VARCHAR(500),              -- First result URL
    
    -- Cost tracking
    cost DECIMAL(10, 4),                      -- $0.009 per query (AI Base)
    api_tier VARCHAR(50),                     -- 'free', 'base_ai', 'pro_ai'
    
    -- Caching
    cache_enabled TINYINT DEFAULT 1,          -- Can this be reused?
    cache_expires_at TIMESTAMP,               -- Expires in 30 days
    cache_hit TINYINT DEFAULT 0,              -- 1 if used cached result
    
    -- Metadata
    search_category VARCHAR(100),             -- 'features', 'pricing', 'competitors', 'news'
    agent_decision TEXT,                      -- Why agent chose this search
    relevance_score DECIMAL(3,2),             -- Agent rating 0-1.0 of relevance
    status VARCHAR(30) DEFAULT 'completed',   -- completed, failed, rate_limited
    error_message TEXT,                       -- If failed
    
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    
    KEY idx_post (post_id),
    KEY idx_query (search_query),
    KEY idx_cache_expires (cache_expires_at),
    KEY idx_status (status),
    KEY idx_created (created_at)
);

Table: wp_agentic_citations

Tracks every citation [1], [2], [3]... in the article with source URL.

CREATE TABLE wp_agentic_citations (
    id BIGINT AUTO_INCREMENT PRIMARY KEY,
    post_id BIGINT NOT NULL,
    
    -- Citation numbering
    citation_number INT NOT NULL,             -- [1], [2], [3]...
    citation_text VARCHAR(500),               -- Text that was cited
    context_excerpt TEXT,                     -- Sentence containing citation
    
    -- Source information
    search_id BIGINT,                         -- Reference to wp_agentic_searches
    source_url VARCHAR(500) NOT NULL,         -- Full URL of source
    source_title VARCHAR(255),                -- "N8n Pricing | Official Website"
    source_domain VARCHAR(100),               -- "n8n.io"
    source_type VARCHAR(50),                  -- 'official_website', 'blog', 'news', 'doc'
    
    -- Citation credibility
    source_authority INT,                     -- 1-100 (domain authority estimate)
    result_position INT,                      -- Was this the 1st, 5th, 10th result?
    snippet_match_score DECIMAL(3,2),         -- % match with snippet
    
    -- Article placement
    article_section VARCHAR(100),             -- "Introduction", "Features", "Pricing"
    paragraph_number INT,                     -- Which paragraph in section
    sentence_number INT,                      -- Which sentence in paragraph
    inline_position INT,                      -- Position of [N] in sentence
    
    -- Metadata
    added_by VARCHAR(50),                     -- 'agent_automatic' or 'user_manual'
    verified TINYINT DEFAULT 0,               -- User confirmed source is correct
    is_required TINYINT DEFAULT 1,            -- Essential citation vs optional
    
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    
    KEY idx_post (post_id),
    KEY idx_citation_number (post_id, citation_number),
    KEY idx_source_domain (source_domain),
    KEY idx_created (created_at)
);

Table: wp_agentic_search_cache

Cache layer for frequently searched topics.

CREATE TABLE wp_agentic_search_cache (
    id BIGINT AUTO_INCREMENT PRIMARY KEY,
    
    -- Cache key
    search_query_normalized VARCHAR(500) NOT NULL,   -- Lowercase, trimmed
    search_category VARCHAR(100),                    -- Optional: 'news', 'trends'
    cache_key VARCHAR(64),                           -- SHA1 hash of query
    
    -- Cached data
    results_json LONGTEXT,                           -- Complete Brave response
    result_count INT,
    cached_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP,                            -- 30 days from creation
    
    -- Usage tracking
    hit_count INT DEFAULT 0,                         -- Times this cache was used
    cost_saved DECIMAL(10, 4),                       -- Cumulative cost avoided
    
    -- Quality
    quality_score DECIMAL(3,2),                      -- 0-1.0 relevance rating
    
    UNIQUE KEY unique_query_category (search_query_normalized, search_category),
    KEY idx_expires (expires_at),
    KEY idx_hit_count (hit_count)
);

---

Flow 1: Agent Research Planning

How the Agent Decides What to Search

<?php
/**
 * Agent analyzes topic and creates search strategy
 * 
 * Input: Topic, target audience, depth level
 * Output: Array of searches to execute
 */

class ResearchPlanner {
    
    public static function plan_searches($topic, $depth = 'medium', $audience = 'general') {
        // Example: Topic = "N8n automation platform"
        
        $searches = [];
        
        // Tier 1: Core knowledge (always needed)
        $searches[] = [
            'query' => "$topic what is",
            'category' => 'definition',
            'priority' => 'critical',
            'intent' => 'Define what N8n is, core features'
        ];
        
        if ($depth === 'medium' || $depth === 'deep') {
            // Tier 2: Current state & news
            $searches[] = [
                'query' => "$topic latest news 2024",
                'category' => 'news',
                'priority' => 'high',
                'intent' => 'Recent developments, updates'
            ];
            
            // Tier 3: Pricing & comparison
            $searches[] = [
                'query' => "$topic pricing plans features",
                'category' => 'pricing',
                'priority' => 'high',
                'intent' => 'Current pricing, plan comparison'
            ];
            
            // Tier 4: Alternatives & comparison
            $searches[] = [
                'query' => "$topic vs alternatives competitors",
                'category' => 'comparison',
                'priority' => 'medium',
                'intent' => 'How does it compare to Zapier, Make, etc'
            ];
        }
        
        if ($depth === 'deep') {
            // Tier 5: Use cases & case studies
            $searches[] = [
                'query' => "$topic use cases examples case studies",
                'category' => 'examples',
                'priority' => 'medium',
                'intent' => 'Real-world examples and success stories'
            ];
            
            // Tier 6: Technical implementation
            $searches[] = [
                'query' => "$topic documentation api integration",
                'category' => 'technical',
                'priority' => 'medium',
                'intent' => 'How to implement, API docs, guides'
            ];
        }
        
        return $searches;
    }
}

---

Flow 2: Brave Search API Integration

Backend: Brave Search Client

<?php
/**
 * Wrapper for Brave Search API
 * Handles authentication, retries, error handling
 */

class BraveSearchClient {
    
    private $api_key;
    private $api_base = 'https://api.search.brave.com/res/v1';
    private $rate_limiter;
    
    public function __construct($api_key = null) {
        $this->api_key = $api_key ?? get_option('agentic_brave_api_key');
        $this->rate_limiter = new RateLimiter();
    }
    
    /**
     * Main search method
     * Handles caching, rate limiting, cost tracking
     */
    public function search($query, $options = []) {
        // 1. Check cache first
        $cached = $this->check_cache($query, $options['category'] ?? null);
        if ($cached) {
            return [
                'results' => $cached['results_json'],
                'from_cache' => true,
                'cache_age_hours' => $cached['age']
            ];
        }
        
        // 2. Rate limit check
        if (!$this->rate_limiter->allow_request($this->api_key)) {
            return new WP_Error(
                'rate_limit_exceeded',
                'Brave Search API rate limit exceeded. Try again in ' . 
                $this->rate_limiter->get_reset_time() . ' seconds'
            );
        }
        
        // 3. Call Brave API
        $start_time = microtime(true);
        $response = $this->call_brave_api($query, $options);
        $execution_time = microtime(true) - $start_time;
        
        if (is_wp_error($response)) {
            return $response;
        }
        
        // 4. Store search in database
        $cost = $this->calculate_cost($this->get_tier());
        $search_id = $this->store_search(
            $query,
            $response,
            $cost,
            $options
        );
        
        // 5. Cache the results
        $this->cache_results($query, $response, $options);
        
        // 6. Track cost
        update_option(
            'agentic_brave_total_cost',
            (float)get_option('agentic_brave_total_cost', 0) + $cost
        );
        
        return [
            'results' => $response,
            'from_cache' => false,
            'search_id' => $search_id,
            'cost' => $cost,
            'execution_time' => $execution_time,
            'result_count' => count($response['web']['results'] ?? [])
        ];
    }
    
    /**
     * Call Brave Search API
     */
    private function call_brave_api($query, $options = []) {
        $endpoint = $this->api_base . '/web/search';
        
        $params = [
            'q' => $query,
            'count' => $options['count'] ?? 10,
            'safesearch' => 'moderate',
            'search_lang' => $options['language'] ?? 'en',
            'country' => $options['country'] ?? 'US',
        ];
        
        // Optional: Use Search Goggles for custom ranking
        if (!empty($options['goggle'])) {
            $params['goggles_id'] = $options['goggle'];
        }
        
        $response = wp_remote_get(
            add_query_arg($params, $endpoint),
            [
                'headers' => [
                    'X-Subscription-Token' => $this->api_key,
                    'Accept' => 'application/json'
                ],
                'timeout' => 30
            ]
        );
        
        if (is_wp_error($response)) {
            return new WP_Error(
                'api_request_failed',
                'Brave Search API request failed: ' . $response->get_error_message()
            );
        }
        
        $status = wp_remote_retrieve_response_code($response);
        $body = json_decode(wp_remote_retrieve_body($response), true);
        
        // Handle API errors
        if ($status === 429) {
            // Rate limited
            return new WP_Error('rate_limited', 'API rate limit hit');
        } elseif ($status === 401) {
            return new WP_Error('invalid_api_key', 'Invalid Brave API key');
        } elseif ($status !== 200) {
            return new WP_Error(
                'api_error',
                'Brave API returned status ' . $status,
                ['response_body' => $body]
            );
        }
        
        return $body;
    }
    
    /**
     * Check if results are cached
     */
    private function check_cache($query, $category = null) {
        global $wpdb;
        
        $cache_key = sha1(strtolower(trim($query)));
        
        $cached = $wpdb->get_row($wpdb->prepare(
            "SELECT * FROM {$wpdb->prefix}agentic_search_cache
             WHERE cache_key = %s
             AND expires_at > NOW()
             AND (search_category = %s OR %s IS NULL)
             ORDER BY hit_count DESC",
            $cache_key,
            $category,
            $category
        ));
        
        if ($cached) {
            // Update cache stats
            $wpdb->update(
                $wpdb->prefix . 'agentic_search_cache',
                [
                    'hit_count' => $cached->hit_count + 1,
                    'cost_saved' => $cached->cost_saved + 0.009
                ],
                ['id' => $cached->id]
            );
            
            $age_seconds = strtotime('now') - strtotime($cached->cached_at);
            return [
                'results_json' => json_decode($cached->results_json, true),
                'age' => ceil($age_seconds / 3600)
            ];
        }
        
        return null;
    }
    
    /**
     * Store search in database
     */
    private function store_search($query, $response, $cost, $options = []) {
        global $wpdb;
        
        $top_result = null;
        if (!empty($response['web']['results'])) {
            $top_result = $response['web']['results'][0];
        }
        
        $wpdb->insert(
            $wpdb->prefix . 'agentic_searches',
            [
                'post_id' => $options['post_id'] ?? 0,
                'search_query' => $query,
                'search_number' => $options['search_number'] ?? 1,
                'total_searches_for_post' => $options['total_searches'] ?? 1,
                'results_count' => count($response['web']['results'] ?? []),
                'results_json' => wp_json_encode($response),
                'top_result_title' => $top_result['title'] ?? null,
                'top_result_url' => $top_result['url'] ?? null,
                'cost' => $cost,
                'api_tier' => $this->get_tier(),
                'search_category' => $options['category'] ?? 'general',
                'status' => 'completed'
            ]
        );
        
        return $wpdb->insert_id;
    }
    
    /**
     * Cache search results for 30 days
     */
    private function cache_results($query, $response, $options = []) {
        global $wpdb;
        
        $cache_key = sha1(strtolower(trim($query)));
        
        $wpdb->insert(
            $wpdb->prefix . 'agentic_search_cache',
            [
                'search_query_normalized' => strtolower(trim($query)),
                'search_category' => $options['category'] ?? null,
                'cache_key' => $cache_key,
                'results_json' => wp_json_encode($response),
                'result_count' => count($response['web']['results'] ?? []),
                'expires_at' => gmdate('Y-m-d H:i:s', strtotime('+30 days')),
                'quality_score' => 0.9  // Agent can rate later
            ],
            ['%s', '%s', '%s', '%s', '%d', '%s', '%f']
        );
    }
    
    /**
     * Get current API tier
     */
    private function get_tier() {
        return get_option('agentic_brave_api_tier', 'base_ai');
    }
    
    /**
     * Calculate cost per request
     */
    private function calculate_cost($tier) {
        $tiers = [
            'free' => 0,
            'base_ai' => 0.005,      // $5 per 1000
            'pro_ai' => 0.009        // $9 per 1000
        ];
        
        return $tiers[$tier] ?? 0;
    }
}

---

Flow 3: Citation Management

Agent: Extract and Number Citations

<?php
/**
 * After agent writes article, extract citations
 * Number them [1], [2], [3]...
 * Match each to a source
 */

class CitationManager {
    
    public static function process_citations(
        $post_id,
        $article_content,
        $search_results,
        $searches_performed
    ) {
        // 1. Parse article for citation markers [SOURCE_X]
        // Agent writes: "N8n is workflow automation [n8n_official_docs]"
        
        $citations = [];
        $citation_number = 1;
        
        // Regex: Find [reference_marker] in article
        $pattern = '/\[([a-z0-9_]+)\]/i';
        preg_match_all($pattern, $article_content, $matches);
        
        foreach ($matches[1] as $source_marker) {
            // 2. Find matching source in search results
            $source_info = self::find_source_by_marker(
                $source_marker,
                $search_results
            );
            
            if (!$source_info) {
                // Source not found - log warning
                error_log("Citation source not found: $source_marker");
                continue;
            }
            
            // 3. Store citation record
            $citation_id = self::store_citation(
                $post_id,
                $citation_number,
                $source_info,
                $source_marker
            );
            
            // 4. Replace [source_marker] with [N] in content
            $article_content = str_replace(
                "[$source_marker]",
                "[$citation_number]",
                $article_content
            );
            
            $citations[] = [
                'number' => $citation_number,
                'source' => $source_info
            ];
            
            $citation_number++;
        }
        
        // 5. Add References section to article
        $references_section = self::generate_references_section($citations);
        $article_content .= $references_section;
        
        return [
            'content' => $article_content,
            'citations' => $citations,
            'citation_count' => count($citations)
        ];
    }
    
    /**
     * Find source in search results by marker
     */
    private static function find_source_by_marker($marker, $search_results) {
        // Examples:
        // "n8n_official_docs" → Find URL from n8n.io/docs search
        // "zapier_pricing" → Find from "N8n vs Zapier" search result
        
        // Parse marker to understand what it's looking for
        $parts = explode('_', $marker);
        $topic = $parts[0];
        $type = $parts[1] ?? null;
        
        foreach ($search_results as $search) {
            if (empty($search['results']['web']['results'])) {
                continue;
            }
            
            foreach ($search['results']['web']['results'] as $result) {
                $domain = parse_url($result['url'], PHP_URL_HOST);
                
                // Match: does URL match the topic?
                if (stripos($domain, $topic) !== false ||
                    stripos($result['title'], $topic) !== false) {
                    
                    return [
                        'url' => $result['url'],
                        'title' => $result['title'],
                        'domain' => $domain,
                        'snippet' => $result['snippet'] ?? '',
                        'description' => $result['description'] ?? '',
                        'position' => array_search($result, $search['results']['web']['results'])
                    ];
                }
            }
        }
        
        return null;
    }
    
    /**
     * Store citation in database
     */
    private static function store_citation(
        $post_id,
        $citation_number,
        $source_info,
        $source_marker
    ) {
        global $wpdb;
        
        // Detect source type
        $source_type = self::detect_source_type($source_info['domain']);
        
        $wpdb->insert(
            $wpdb->prefix . 'agentic_citations',
            [
                'post_id' => $post_id,
                'citation_number' => $citation_number,
                'source_url' => $source_info['url'],
                'source_title' => $source_info['title'],
                'source_domain' => $source_info['domain'],
                'source_type' => $source_type,
                'result_position' => $source_info['position'],
                'added_by' => 'agent_automatic',
                'created_at' => current_time('mysql')
            ]
        );
        
        return $wpdb->insert_id;
    }
    
    /**
     * Generate References section
     */
    private static function generate_references_section($citations) {
        $html = '<h2>References</h2><ol>';
        
        foreach ($citations as $citation) {
            $source = $citation['source'];
            $html .= sprintf(
                '<li><a href="%s" target="_blank">%s</a> - %s</li>',
                esc_url($source['url']),
                esc_html($source['title']),
                esc_html($source['domain'])
            );
        }
        
        $html .= '</ol>';
        
        return $html;
    }
    
    /**
     * Classify source credibility
     */
    private static function detect_source_type($domain) {
        if (stripos($domain, 'docs.') === 0 || stripos($domain, '.io') === 0) {
            return 'official_documentation';
        } elseif (stripos($domain, 'blog.') === 0) {
            return 'official_blog';
        } elseif (in_array($domain, ['medium.com', 'dev.to', 'hashnode.com'])) {
            return 'tech_blog';
        } elseif (in_array($domain, ['techcrunch.com', 'theverge.com', 'forbes.com'])) {
            return 'news';
        } elseif (stripos($domain, 'github.com') === 0) {
            return 'github';
        } else {
            return 'general_web';
        }
    }
}

---

Flow 4: Search Result Caching

Cache Strategy

Why caching is critical:

• A single article generation might trigger 3-5 searches
• Multiple articles on similar topics (e.g., "Automation tools") will search same keywords
• Brave API charges per query: $5-9 per 1,000 queries
• Caching can reduce costs by 40-60%

Cache rules:

• Cache all successful searches for 30 days
• Reuse cache for identical queries (case-insensitive, trimmed)
• Group by category (pricing, news, features) for better relevance
• Track cache hits for analytics

Cache Implementation

<?php
class SearchCacheManager {
    
    /**
     * Get or fetch search results
     * Returns cached if valid, otherwise fetches new
     */
    public static function get_or_search($query, $options = []) {
        $cache_key = self::generate_cache_key($query, $options['category'] ?? null);
        
        // Check cache
        $cached = self::get_cached_result($cache_key);
        if ($cached && !self::is_cache_expired($cached)) {
            return [
                'source' => 'cache',
                'results' => $cached,
                'age_days' => self::get_cache_age_days($cached)
            ];
        }
        
        // Fetch fresh
        $client = new BraveSearchClient();
        $fresh = $client->search($query, $options);
        
        return [
            'source' => 'api',
            'results' => $fresh
        ];
    }
    
    /**
     * Cleanup expired cache entries
     * Run daily via cron
     */
    public static function cleanup_expired_cache() {
        global $wpdb;
        
        $deleted = $wpdb->query(
            "DELETE FROM {$wpdb->prefix}agentic_search_cache
             WHERE expires_at < NOW()"
        );
        
        error_log("Cleaned up $deleted expired search cache entries");
    }
    
    /**
     * Manual cache invalidation (if search results become stale)
     */
    public static function invalidate_cache($query, $category = null) {
        global $wpdb;
        
        $cache_key = self::generate_cache_key($query, $category);
        
        $wpdb->update(
            $wpdb->prefix . 'agentic_search_cache',
            ['expires_at' => current_time('mysql')],  // Set to now (expired)
            ['cache_key' => $cache_key]
        );
        
        return true;
    }
    
    private static function generate_cache_key($query, $category = null) {
        $normalized = strtolower(trim($query));
        return sha1($normalized . ($category ? "_$category" : ''));
    }
}

// Schedule cache cleanup daily
if (!wp_next_scheduled('agentic_search_cache_cleanup')) {
    wp_schedule_event(
        time(),
        'daily',
        'agentic_search_cache_cleanup'
    );
}

add_action('agentic_search_cache_cleanup', [
    'SearchCacheManager',
    'cleanup_expired_cache'
]);

---

Flow 5: Admin Dashboard & Analytics

Search Analytics Tab

Plugin Settings → Research & Citations → Search Analytics

┌────────────────────────────────────────────────────────┐
│ Search Performance & Cost Analytics                    │
├────────────────────────────────────────────────────────┤
│                                                         │
│ COST SUMMARY                                            │
│ ─────────────────────────────────────────────────────  │
│ Total searches: 156                                     │
│ Cached hits: 89 (57%)                                   │
│ Fresh API calls: 67 (43%)                               │
│ Total cost: $0.60 (would be $1.40 without cache)        │
│ Monthly budget: $50.00                                  │
│ Budget used: 1.2% ✓                                     │
│                                                         │
│ TOP SEARCHES (by frequency)                             │
│ ─────────────────────────────────────────────────────  │
│ 1. "n8n automation features" → 23 hits (13 cached)      │
│    Cost: $0.09 | Last: Jan 28, 2:30 PM                 │
│                                                         │
│ 2. "workflow automation comparison" → 18 hits (11 cache│
│    Cost: $0.09 | Last: Jan 27, 5:15 PM                 │
│                                                         │
│ 3. "zapier pricing 2024" → 12 hits (7 cached)          │
│    Cost: $0.05 | Last: Jan 27, 1:20 PM                 │
│                                                         │
│ CACHE PERFORMANCE                                       │
│ ─────────────────────────────────────────────────────  │
│ Cache hit rate: 57%                                     │
│ Cache age (avg): 8 days                                 │
│ Cost saved by cache: $0.80 (57% reduction)              │
│                                                         │
│ [Clear Cache] [Invalidate > 30 days] [Download Report] │
│                                                         │
└────────────────────────────────────────────────────────┘

---

REST API Endpoints

Public REST Endpoints

/**
 * POST /wp-json/agentic-writer/v1/search
 * Perform a web search using Brave API
 * 
 * Request:
 * {
 *   "query": "N8n automation features",
 *   "category": "features",
 *   "count": 10,
 *   "country": "US"
 * }
 * 
 * Response:
 * {
 *   "success": true,
 *   "results": [...],
 *   "from_cache": false,
 *   "cost": 0.009,
 *   "result_count": 10
 * }
 */

register_rest_route('agentic-writer/v1', '/search', [
    'methods' => 'POST',
    'callback' => 'agentic_writer_rest_search',
    'permission_callback' => 'is_user_logged_in',
    'args' => [
        'query' => ['required' => true, 'type' => 'string'],
        'category' => ['type' => 'string'],
        'count' => ['type' => 'integer', 'default' => 10]
    ]
]);

function agentic_writer_rest_search($request) {
    $query = $request->get_param('query');
    $client = new BraveSearchClient();
    
    return $client->search($query, [
        'category' => $request->get_param('category'),
        'count' => $request->get_param('count'),
        'post_id' => $request->get_param('post_id')
    ]);
}

/**
 * GET /wp-json/agentic-writer/v1/searches?post_id=123
 * List all searches for a post with citations
 */

register_rest_route('agentic-writer/v1', '/searches', [
    'methods' => 'GET',
    'callback' => 'agentic_writer_rest_list_searches',
    'permission_callback' => 'is_user_logged_in',
    'args' => ['post_id' => ['required' => true, 'type' => 'integer']]
]);

function agentic_writer_rest_list_searches($request) {
    global $wpdb;
    
    $post_id = $request->get_param('post_id');
    
    if (!current_user_can('edit_post', $post_id)) {
        return new WP_Error('unauthorized', 'Not allowed', ['status' => 403]);
    }
    
    $searches = $wpdb->get_results($wpdb->prepare(
        "SELECT * FROM {$wpdb->prefix}agentic_searches
         WHERE post_id = %d
         ORDER BY created_at DESC",
        $post_id
    ));
    
    $citations = $wpdb->get_results($wpdb->prepare(
        "SELECT * FROM {$wpdb->prefix}agentic_citations
         WHERE post_id = %d
         ORDER BY citation_number ASC",
        $post_id
    ));
    
    return [
        'searches' => $searches,
        'citations' => $citations,
        'total_cost' => array_sum(array_map(function($s) {
            return floatval($s->cost);
        }, $searches))
    ];
}

---

Configuration & Settings

Settings Panel Integration

Add new tab to WP Agentic Writer settings:

'brave_search_settings' => [
    'brave_api_key' => '',                    // Required
    'brave_api_tier' => 'base_ai',            // free, base_ai, pro_ai
    'brave_search_enabled' => true,           // Toggle feature on/off
    'enable_search_caching' => true,          // Cache results
    'cache_duration_days' => 30,              // How long to keep cache
    'auto_search_enabled' => true,            // Auto-search during generation
    'search_count_per_article' => 3,          // How many searches per article
    'include_citations' => true,              // Add [1], [2]... to article
    'include_references_section' => true,     // Add References section
    'monthly_budget_limit' => 50.00,          // Dollar limit per month
    'budget_alert_threshold' => 75,           // Alert at 75% of budget
    'auto_invalidate_cache_older_than_days' => 45,  // Auto-cleanup old cache
    'search_result_quality_threshold' => 0.6, // Minimum relevance score
]

---

Cost Optimization & Budget Management

Real-time Cost Tracking

<?php
class BraveSearchCostTracker {
    
    /**
     * Log cost and check budget
     */
    public static function track_cost($cost, $post_id = null) {
        $monthly_cost = self::get_monthly_cost();
        $budget_limit = floatval(get_option('agentic_brave_budget_limit', 50.00));
        
        update_option(
            'agentic_brave_monthly_cost',
            $monthly_cost + $cost
        );
        
        $new_cost = $monthly_cost + $cost;
        $percentage = ($new_cost / $budget_limit) * 100;
        
        // Alert if threshold exceeded
        if ($percentage >= floatval(get_option('agentic_brave_budget_alert', 75))) {
            do_action('agentic_brave_budget_alert', [
                'monthly_cost' => $new_cost,
                'budget_limit' => $budget_limit,
                'percentage' => $percentage
            ]);
        }
        
        // Block searches if over budget
        if ($new_cost >= $budget_limit) {
            return new WP_Error(
                'budget_exceeded',
                sprintf(
                    'Monthly budget of $%.2f exceeded (current: $%.2f)',
                    $budget_limit,
                    $new_cost
                )
            );
        }
        
        return true;
    }
    
    /**
     * Get current month's total cost
     */
    private static function get_monthly_cost() {
        global $wpdb;
        
        $result = $wpdb->get_var(
            "SELECT COALESCE(SUM(cost), 0) FROM {$wpdb->prefix}agentic_searches
             WHERE MONTH(created_at) = MONTH(NOW())
             AND YEAR(created_at) = YEAR(NOW())"
        );
        
        return floatval($result);
    }
    
    /**
     * Generate cost report
     */
    public static function get_cost_report() {
        global $wpdb;
        
        return [
            'today' => self::get_period_cost('today'),
            'week' => self::get_period_cost('week'),
            'month' => self::get_period_cost('month'),
            'all_time' => self::get_all_time_cost(),
            'top_searches' => self::get_top_searches_by_cost(),
            'cache_savings' => self::get_cache_savings()
        ];
    }
    
    /**
     * Calculate how much cache saved
     */
    private static function get_cache_savings() {
        global $wpdb;
        
        return $wpdb->get_var(
            "SELECT COALESCE(SUM(cost_saved), 0) FROM {$wpdb->prefix}agentic_search_cache"
        );
    }
}

// Hook for budget alerts
add_action('agentic_brave_budget_alert', function($data) {
    // Send admin email
    wp_mail(
        get_option('admin_email'),
        'WP Agentic Writer: Budget Alert',
        sprintf(
            "Brave Search API budget is at %.0f%% ($%.2f / $%.2f)\n\nGo to Settings to adjust limits.",
            $data['percentage'],
            $data['monthly_cost'],
            $data['budget_limit']
        )
    );
});

---

Implementation Checklist

Phase 1: Core Integration (Week 1-2)

• Create database tables: wp_agentic_searches, wp_agentic_citations, wp_agentic_search_cache
• Build BraveSearchClient class with API authentication
• Implement basic search → cache → cost tracking flow
• Create REST endpoint: POST /search
• Add settings panel for API key & tier selection
• Test with sample searches

Phase 2: Agent Integration (Week 2-3)

• Build ResearchPlanner (automatic search strategy)
• Integrate searches into article generation workflow
• Build CitationManager (extract, number, reference)
• Add citations to generated articles ([1], [2]... References)
• Test end-to-end: Topic → Searches → Article with Citations

Phase 3: Caching & Optimization (Week 3-4)

• Implement SearchCacheManager with 30-day expiry
• Add cache hit tracking & cost savings calculation
• Build cache cleanup cron job
• Implement cache invalidation endpoint
• Test cache reuse scenarios

Phase 4: Admin & Analytics (Week 4-5)

• Build admin dashboard with Search Analytics tab
• Add cost tracking & monthly budget management
• Create search performance reports
• Add budget alert system (email)
• Build cache management UI (view, invalidate, cleanup)

Phase 5: Security & Polish (Week 5)

• Add rate limiting per user
• Sanitize search queries
• Add audit logging (who searched what, when)
• Implement permission checks
• Add error handling for API failures
• Test with high volume searches

---

Next Steps

1. Set up Brave Search API account (free tier first)
2. Create database migrations for the three new tables
3. Begin Phase 1 implementation (BraveSearchClient)
4. Test API connectivity and response parsing
5. Move to Phase 2 once API is reliable