# WP Agentic Writer: Brave Search API Integration Guide ## Executive Summary This document defines the **complete integration** of **Brave Search API** into WP Agentic Writer to enhance article generation with real-time web research, citations, and fact-grounding. **Key principle:** During article generation, the writing agent calls Brave Search API to fetch current data, verify facts, gather citations, and ground responses in real sources. All search results are logged, cached, and attributed. --- ## Table of Contents 1. [Overview & Architecture](#overview--architecture) 2. [Brave Search API Fundamentals](#brave-search-api-fundamentals) 3. [Data Model & Database Schema](#data-model--database-schema) 4. [Flow 1: Agent Research Planning](#flow-1-agent-research-planning) 5. [Flow 2: Brave Search API Integration](#flow-2-brave-search-api-integration) 6. [Flow 3: Citation Management](#flow-3-citation-management) 7. [Flow 4: Search Result Caching](#flow-4-search-result-caching) 8. [Flow 5: Admin Dashboard & Analytics](#flow-5-admin-dashboard--analytics) 9. [REST API Endpoints](#rest-api-endpoints) 10. [Configuration & Settings](#configuration--settings) 11. [Cost Optimization & Budget Management](#cost-optimization--budget-management) 12. [Implementation Checklist](#implementation-checklist) --- ## Overview & Architecture ### Core Concept ``` ┌─────────────────────────────────────────────────────────────────┐ │ USER: "Write article about N8n automation workflows" │ └────────────────────┬────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ │ WRITING AGENT receives request │ │ - Analyzes: topic, target audience, depth needed │ │ - Plans research: "Need current N8n features, pricing, users" │ └────────────────────┬────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ │ AGENT → Brave Search API (Multiple searches) │ │ │ │ Search 1: "N8n automation platform 2024" │ │ Search 2: "N8n pricing plans features" │ │ Search 3: "N8n vs Zapier Make comparison" │ │ Search 4: "N8n self-hosted deployment guide" │ │ │ │ Each search: Get 10-15 results with snippets + URLs │ └────────────────────┬────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ │ BACKEND: Store searches in DB │ │ │ │ wp_agentic_searches table: │ │ - post_id, search_query, results (serialized), cost │ │ - cache_enabled (reuse for 30 days) │ │ │ │ wp_agentic_citations table: │ │ - post_id, citation_text, url, source_title, position │ │ - [1], [2], [3] numbered references │ └────────────────────┬────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ │ AGENT: Generates article WITH research │ │ │ │ "N8n is an open-source workflow automation platform[1] │ │ enabling teams to automate tasks[2]. In 2024, N8n added[3]... │ │ │ │ Key features include visual workflow builder[1], │ │ 500+ integrations[2], and self-hosting options[3]." │ │ │ │ Returns: Article blocks + citations (with sources) │ └────────────────────┬────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ │ PLUGIN: Create Gutenberg post with: │ │ - Article content │ │ - Inline citations [1], [2], [3] │ │ - References section at end (with URLs) │ │ - Search context sidebar (optional) │ └────────────────────┬────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────────────────┐ │ ADMIN DASHBOARD: "Generated Searches" │ │ │ │ Show: │ │ - All searches per post + results count │ │ - Cost per search ($0.009 per query for AI tier) │ │ - Cache status (reusable until expiry) │ │ - Citation coverage (% of facts cited) │ └─────────────────────────────────────────────────────────────────┘ ``` --- ## Brave Search API Fundamentals ### API Plans & Pricing (2024) **Free Tier (for testing):** - 1 query/second - 2,000 queries/month (~$0) - Great for development & testing **Data for AI - Base ($5 per 1,000 requests):** - 20 queries/second - Up to 20M queries/month - Up to 5 snippets per result - Extra alternate snippets for AI - Rights to use in AI apps **Data for AI - Pro ($9 per 1,000 requests):** - 50 queries/second - Unlimited queries/month - Same features as Base + schema-enriched results - Recommended for production AI apps ### Key Advantages Over Alternatives | Feature | Brave Search | Google Search | Bing Search | |---------|-----------|---------|---------| | **Cost** | $3-9 CPM | ~$20 CPM | ~$15 CPM | | **Independent Index** | Yes (30B+ pages) | No (proprietary) | No (proprietary) | | **Privacy** | Yes (no tracking) | No | No | | **AI Rights** | Explicit in plan | Unclear | Unclear | | **Fresh Data** | 100M updates/day | Daily | Daily | | **Latency** | ~2-3s average | Fast | Fast | --- ## Data Model & Database Schema ### Table: `wp_agentic_searches` Stores all Brave Search API calls made during article generation. ```sql CREATE TABLE wp_agentic_searches ( id BIGINT AUTO_INCREMENT PRIMARY KEY, post_id BIGINT NOT NULL, -- Search details search_query VARCHAR(500) NOT NULL, -- "N8n automation features" search_number INT, -- 1st, 2nd, 3rd search total_searches_for_post INT, -- "Need 3 searches for this post" -- Brave API response results_count INT, -- How many results returned results_json LONGTEXT, -- Serialized JSON from Brave top_result_title VARCHAR(255), -- First result title top_result_url VARCHAR(500), -- First result URL -- Cost tracking cost DECIMAL(10, 4), -- $0.009 per query (AI Base) api_tier VARCHAR(50), -- 'free', 'base_ai', 'pro_ai' -- Caching cache_enabled TINYINT DEFAULT 1, -- Can this be reused? cache_expires_at TIMESTAMP, -- Expires in 30 days cache_hit TINYINT DEFAULT 0, -- 1 if used cached result -- Metadata search_category VARCHAR(100), -- 'features', 'pricing', 'competitors', 'news' agent_decision TEXT, -- Why agent chose this search relevance_score DECIMAL(3,2), -- Agent rating 0-1.0 of relevance status VARCHAR(30) DEFAULT 'completed', -- completed, failed, rate_limited error_message TEXT, -- If failed created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, KEY idx_post (post_id), KEY idx_query (search_query), KEY idx_cache_expires (cache_expires_at), KEY idx_status (status), KEY idx_created (created_at) ); ``` ### Table: `wp_agentic_citations` Tracks every citation [1], [2], [3]... in the article with source URL. ```sql CREATE TABLE wp_agentic_citations ( id BIGINT AUTO_INCREMENT PRIMARY KEY, post_id BIGINT NOT NULL, -- Citation numbering citation_number INT NOT NULL, -- [1], [2], [3]... citation_text VARCHAR(500), -- Text that was cited context_excerpt TEXT, -- Sentence containing citation -- Source information search_id BIGINT, -- Reference to wp_agentic_searches source_url VARCHAR(500) NOT NULL, -- Full URL of source source_title VARCHAR(255), -- "N8n Pricing | Official Website" source_domain VARCHAR(100), -- "n8n.io" source_type VARCHAR(50), -- 'official_website', 'blog', 'news', 'doc' -- Citation credibility source_authority INT, -- 1-100 (domain authority estimate) result_position INT, -- Was this the 1st, 5th, 10th result? snippet_match_score DECIMAL(3,2), -- % match with snippet -- Article placement article_section VARCHAR(100), -- "Introduction", "Features", "Pricing" paragraph_number INT, -- Which paragraph in section sentence_number INT, -- Which sentence in paragraph inline_position INT, -- Position of [N] in sentence -- Metadata added_by VARCHAR(50), -- 'agent_automatic' or 'user_manual' verified TINYINT DEFAULT 0, -- User confirmed source is correct is_required TINYINT DEFAULT 1, -- Essential citation vs optional created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, KEY idx_post (post_id), KEY idx_citation_number (post_id, citation_number), KEY idx_source_domain (source_domain), KEY idx_created (created_at) ); ``` ### Table: `wp_agentic_search_cache` Cache layer for frequently searched topics. ```sql CREATE TABLE wp_agentic_search_cache ( id BIGINT AUTO_INCREMENT PRIMARY KEY, -- Cache key search_query_normalized VARCHAR(500) NOT NULL, -- Lowercase, trimmed search_category VARCHAR(100), -- Optional: 'news', 'trends' cache_key VARCHAR(64), -- SHA1 hash of query -- Cached data results_json LONGTEXT, -- Complete Brave response result_count INT, cached_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, expires_at TIMESTAMP, -- 30 days from creation -- Usage tracking hit_count INT DEFAULT 0, -- Times this cache was used cost_saved DECIMAL(10, 4), -- Cumulative cost avoided -- Quality quality_score DECIMAL(3,2), -- 0-1.0 relevance rating UNIQUE KEY unique_query_category (search_query_normalized, search_category), KEY idx_expires (expires_at), KEY idx_hit_count (hit_count) ); ``` --- ## Flow 1: Agent Research Planning ### How the Agent Decides What to Search ```php "$topic what is", 'category' => 'definition', 'priority' => 'critical', 'intent' => 'Define what N8n is, core features' ]; if ($depth === 'medium' || $depth === 'deep') { // Tier 2: Current state & news $searches[] = [ 'query' => "$topic latest news 2024", 'category' => 'news', 'priority' => 'high', 'intent' => 'Recent developments, updates' ]; // Tier 3: Pricing & comparison $searches[] = [ 'query' => "$topic pricing plans features", 'category' => 'pricing', 'priority' => 'high', 'intent' => 'Current pricing, plan comparison' ]; // Tier 4: Alternatives & comparison $searches[] = [ 'query' => "$topic vs alternatives competitors", 'category' => 'comparison', 'priority' => 'medium', 'intent' => 'How does it compare to Zapier, Make, etc' ]; } if ($depth === 'deep') { // Tier 5: Use cases & case studies $searches[] = [ 'query' => "$topic use cases examples case studies", 'category' => 'examples', 'priority' => 'medium', 'intent' => 'Real-world examples and success stories' ]; // Tier 6: Technical implementation $searches[] = [ 'query' => "$topic documentation api integration", 'category' => 'technical', 'priority' => 'medium', 'intent' => 'How to implement, API docs, guides' ]; } return $searches; } } ``` --- ## Flow 2: Brave Search API Integration ### Backend: Brave Search Client ```php api_key = $api_key ?? get_option('agentic_brave_api_key'); $this->rate_limiter = new RateLimiter(); } /** * Main search method * Handles caching, rate limiting, cost tracking */ public function search($query, $options = []) { // 1. Check cache first $cached = $this->check_cache($query, $options['category'] ?? null); if ($cached) { return [ 'results' => $cached['results_json'], 'from_cache' => true, 'cache_age_hours' => $cached['age'] ]; } // 2. Rate limit check if (!$this->rate_limiter->allow_request($this->api_key)) { return new WP_Error( 'rate_limit_exceeded', 'Brave Search API rate limit exceeded. Try again in ' . $this->rate_limiter->get_reset_time() . ' seconds' ); } // 3. Call Brave API $start_time = microtime(true); $response = $this->call_brave_api($query, $options); $execution_time = microtime(true) - $start_time; if (is_wp_error($response)) { return $response; } // 4. Store search in database $cost = $this->calculate_cost($this->get_tier()); $search_id = $this->store_search( $query, $response, $cost, $options ); // 5. Cache the results $this->cache_results($query, $response, $options); // 6. Track cost update_option( 'agentic_brave_total_cost', (float)get_option('agentic_brave_total_cost', 0) + $cost ); return [ 'results' => $response, 'from_cache' => false, 'search_id' => $search_id, 'cost' => $cost, 'execution_time' => $execution_time, 'result_count' => count($response['web']['results'] ?? []) ]; } /** * Call Brave Search API */ private function call_brave_api($query, $options = []) { $endpoint = $this->api_base . '/web/search'; $params = [ 'q' => $query, 'count' => $options['count'] ?? 10, 'safesearch' => 'moderate', 'search_lang' => $options['language'] ?? 'en', 'country' => $options['country'] ?? 'US', ]; // Optional: Use Search Goggles for custom ranking if (!empty($options['goggle'])) { $params['goggles_id'] = $options['goggle']; } $response = wp_remote_get( add_query_arg($params, $endpoint), [ 'headers' => [ 'X-Subscription-Token' => $this->api_key, 'Accept' => 'application/json' ], 'timeout' => 30 ] ); if (is_wp_error($response)) { return new WP_Error( 'api_request_failed', 'Brave Search API request failed: ' . $response->get_error_message() ); } $status = wp_remote_retrieve_response_code($response); $body = json_decode(wp_remote_retrieve_body($response), true); // Handle API errors if ($status === 429) { // Rate limited return new WP_Error('rate_limited', 'API rate limit hit'); } elseif ($status === 401) { return new WP_Error('invalid_api_key', 'Invalid Brave API key'); } elseif ($status !== 200) { return new WP_Error( 'api_error', 'Brave API returned status ' . $status, ['response_body' => $body] ); } return $body; } /** * Check if results are cached */ private function check_cache($query, $category = null) { global $wpdb; $cache_key = sha1(strtolower(trim($query))); $cached = $wpdb->get_row($wpdb->prepare( "SELECT * FROM {$wpdb->prefix}agentic_search_cache WHERE cache_key = %s AND expires_at > NOW() AND (search_category = %s OR %s IS NULL) ORDER BY hit_count DESC", $cache_key, $category, $category )); if ($cached) { // Update cache stats $wpdb->update( $wpdb->prefix . 'agentic_search_cache', [ 'hit_count' => $cached->hit_count + 1, 'cost_saved' => $cached->cost_saved + 0.009 ], ['id' => $cached->id] ); $age_seconds = strtotime('now') - strtotime($cached->cached_at); return [ 'results_json' => json_decode($cached->results_json, true), 'age' => ceil($age_seconds / 3600) ]; } return null; } /** * Store search in database */ private function store_search($query, $response, $cost, $options = []) { global $wpdb; $top_result = null; if (!empty($response['web']['results'])) { $top_result = $response['web']['results'][0]; } $wpdb->insert( $wpdb->prefix . 'agentic_searches', [ 'post_id' => $options['post_id'] ?? 0, 'search_query' => $query, 'search_number' => $options['search_number'] ?? 1, 'total_searches_for_post' => $options['total_searches'] ?? 1, 'results_count' => count($response['web']['results'] ?? []), 'results_json' => wp_json_encode($response), 'top_result_title' => $top_result['title'] ?? null, 'top_result_url' => $top_result['url'] ?? null, 'cost' => $cost, 'api_tier' => $this->get_tier(), 'search_category' => $options['category'] ?? 'general', 'status' => 'completed' ] ); return $wpdb->insert_id; } /** * Cache search results for 30 days */ private function cache_results($query, $response, $options = []) { global $wpdb; $cache_key = sha1(strtolower(trim($query))); $wpdb->insert( $wpdb->prefix . 'agentic_search_cache', [ 'search_query_normalized' => strtolower(trim($query)), 'search_category' => $options['category'] ?? null, 'cache_key' => $cache_key, 'results_json' => wp_json_encode($response), 'result_count' => count($response['web']['results'] ?? []), 'expires_at' => gmdate('Y-m-d H:i:s', strtotime('+30 days')), 'quality_score' => 0.9 // Agent can rate later ], ['%s', '%s', '%s', '%s', '%d', '%s', '%f'] ); } /** * Get current API tier */ private function get_tier() { return get_option('agentic_brave_api_tier', 'base_ai'); } /** * Calculate cost per request */ private function calculate_cost($tier) { $tiers = [ 'free' => 0, 'base_ai' => 0.005, // $5 per 1000 'pro_ai' => 0.009 // $9 per 1000 ]; return $tiers[$tier] ?? 0; } } ``` --- ## Flow 3: Citation Management ### Agent: Extract and Number Citations ```php $citation_number, 'source' => $source_info ]; $citation_number++; } // 5. Add References section to article $references_section = self::generate_references_section($citations); $article_content .= $references_section; return [ 'content' => $article_content, 'citations' => $citations, 'citation_count' => count($citations) ]; } /** * Find source in search results by marker */ private static function find_source_by_marker($marker, $search_results) { // Examples: // "n8n_official_docs" → Find URL from n8n.io/docs search // "zapier_pricing" → Find from "N8n vs Zapier" search result // Parse marker to understand what it's looking for $parts = explode('_', $marker); $topic = $parts[0]; $type = $parts[1] ?? null; foreach ($search_results as $search) { if (empty($search['results']['web']['results'])) { continue; } foreach ($search['results']['web']['results'] as $result) { $domain = parse_url($result['url'], PHP_URL_HOST); // Match: does URL match the topic? if (stripos($domain, $topic) !== false || stripos($result['title'], $topic) !== false) { return [ 'url' => $result['url'], 'title' => $result['title'], 'domain' => $domain, 'snippet' => $result['snippet'] ?? '', 'description' => $result['description'] ?? '', 'position' => array_search($result, $search['results']['web']['results']) ]; } } } return null; } /** * Store citation in database */ private static function store_citation( $post_id, $citation_number, $source_info, $source_marker ) { global $wpdb; // Detect source type $source_type = self::detect_source_type($source_info['domain']); $wpdb->insert( $wpdb->prefix . 'agentic_citations', [ 'post_id' => $post_id, 'citation_number' => $citation_number, 'source_url' => $source_info['url'], 'source_title' => $source_info['title'], 'source_domain' => $source_info['domain'], 'source_type' => $source_type, 'result_position' => $source_info['position'], 'added_by' => 'agent_automatic', 'created_at' => current_time('mysql') ] ); return $wpdb->insert_id; } /** * Generate References section */ private static function generate_references_section($citations) { $html = '