A sophisticated AI agent that performs Generative Engine Optimization for websites using LangGraph, OpenAI, and Jina's reranker API. The agent implements a proper agentic workflow that iteratively improves website content and tracks relevance score improvements based on target keywords.
- LangGraph Agentic Workflow: Uses LangGraph to create a structured, stateful workflow with proper error handling and conditional logic
- Iterative Optimization: Uses LangChain for LLM calls within a LangGraph workflow that optimizes website content step by step
- Multiple Optimization Types: Title, meta description, headings, content, internal linking, and schema markup
- Score Tracking: Monitors relevance scores using Jina's reranker API
- Threshold-based Improvements: Only keeps changes that meet improvement thresholds
- HTML Processing: Converts HTML to LLM-readable format for accurate scoring
- Backup System: Automatically backs up original content
- History Tracking: Saves optimization history and results
- Error Handling: Comprehensive error handling with graceful fallbacks
final_code3/
├── config.py # Configuration and prompts
├── geo_agent.py # Original GEO AI agent (class-based)
├── langgraph_geo_agent.py # Basic LangGraph GEO agent
├── langgraph_geo_agent_enhanced.py # Enhanced LangGraph GEO agent with full integration
├── run_langgraph_geo.py # Runner script for LangGraph agents
├── html_utils.py # HTML processing utilities
├── searcher.py # Website search functionality
├── webpage_reader.py # LLM-readable content extraction
├── reranker_api.py # Jina reranker API functions
├── website_similarity_analyzer.py # Website similarity analysis
├── query_processor.py # Query processing pipeline
├── requirements.txt # Python dependencies
└── README.md # This file
- Install dependencies:
pip install -r requirements.txt- Set up environment variables:
export OPENAI_API_KEY="your_openai_api_key_here"import asyncio
from langgraph_geo_agent_enhanced import EnhancedLangGraphGEOAgent
async def main():
# Initialize the enhanced LangGraph agent
agent = EnhancedLangGraphGEOAgent()
# Optimize a website
results = await agent.optimize_website(
query="AI search engine optimization",
target_url="https://example.com" # Optional
)
if results["success"]:
print(f"Initial Score: {results['initial_score']:.3f}")
print(f"Final Score: {results['final_score']:.3f}")
print(f"Improvement: {results['improvement']:.3f}")
print(f"Iterations: {results['iterations']}")
# Run the agent
asyncio.run(main())python run_langgraph_geo.pyThis provides an interactive interface to run the LangGraph agent with options for:
- Single optimization
- Demo optimizations
- Interactive mode
from geo_agent import GEOAgent
import os
# Initialize the agent
agent = GEOAgent(os.getenv("OPENAI_API_KEY"))
# Optimize a website
results = agent.optimize_website(
website_url="https://www.hyperbots.com",
target_keywords="AI agents automation",
original_html="<html>...</html>" # Optional: provide HTML directly
)from geo_agent import GEOAgent
from html_utils import fetch_website_html
import os
# Initialize the agent
agent = GEOAgent(os.getenv("OPENAI_API_KEY"))
# Fetch website HTML
html = fetch_website_html("https://www.hyperbots.com")
if html:
# Optimize the website
results = agent.optimize_website(
website_url="https://www.hyperbots.com",
target_keywords="AI agents automation",
original_html=html
)
print(f"Original Score: {results['original_score']:.4f}")
print(f"Final Score: {results['final_score']:.4f}")
print(f"Total Improvement: {results['total_improvement']:.2%}")The agent performs these optimization steps in order:
- Title Optimization (5% threshold)
- Meta Description (3% threshold)
- Heading Structure (4% threshold)
- Content Optimization (8% threshold)
- Internal Linking (2% threshold)
- Schema Markup (3% threshold)
AGENT_CONFIG = {
"llm_model": "gpt-4",
"temperature": 0.7,
"max_tokens": 2000,
"improvement_threshold": 0.02, # 2% minimum improvement
"max_iterations": 10,
"convergence_threshold": 0.01, # Stop if improvement < 1%
"backup_original": True,
"save_optimization_history": True
}The LangGraph agent implements a structured workflow with the following steps:
- Input Validation: Validates query and API keys
- Website Fetching: Retrieves target website HTML
- Related Site Search: Finds and analyzes related websites for context
- Content Extraction: Extracts LLM-readable content from related sites
- Initial Scoring: Gets baseline relevance score
- Content Analysis: Analyzes content and generates optimization recommendations
- Content Optimization: Applies LLM-based optimizations using LangChain
- Improvement Evaluation: Scores the optimized content
- Conditional Iteration: Continues optimization if improvement threshold is met
- Results Saving: Saves optimized HTML and detailed history
validate_input → fetch_target_website → search_related_sites → extract_related_content
↓
get_initial_score → analyze_content → optimize_content → evaluate_improvement
↓
[continue] ← should_continue → [finish] → save_results → END
The workflow includes comprehensive error handling:
- Input validation errors
- Network failures
- API timeouts
- LLM response parsing errors
- Graceful fallbacks for missing APIs
The agent creates several output files:
website_backups/: Original HTML backupsoptimized_content/: Optimized HTML filesgeo_optimization_history.json: Detailed optimization historyllm_content/: LLM-readable content files
🚀 GENERATIVE ENGINE OPTIMIZATION (GEO) AGENT
============================================================
Website: https://www.hyperbots.com
Target Keywords: AI agents automation
📊 Getting initial relevance score...
Initial relevance score: 0.4521
🔄 Optimizing: Optimize page title for SEO
Attempt 1/3
Current score: 0.4521
New score: 0.4876
Improvement: 7.85%
✅ Improvement meets threshold (5.0%)
🔄 Optimizing: Optimize meta description
Attempt 1/3
Current score: 0.4876
New score: 0.5123
Improvement: 5.07%
✅ Improvement meets threshold (3.0%)
✅ OPTIMIZATION COMPLETE
============================================================
Original Score: 0.4521
Final Score: 0.5123
Total Improvement: 13.31%
Iterations: 1
Optimized HTML saved to: optimized_content/optimized_https___www_hyperbots_com_20241201_143022.html
- Add prompt to
GEO_PROMPTSinconfig.py - Add step configuration to
OPTIMIZATION_STEPS - Implement optimization logic in
apply_optimization()method
Modify the threshold values in OPTIMIZATION_STEPS to make the agent more or less strict about improvements.
Update AGENT_CONFIG["llm_model"] to use different OpenAI models.
The agent includes comprehensive error handling for:
- API failures
- HTML parsing errors
- Network timeouts
- Invalid responses
The agent tracks:
- Relevance score improvements
- Optimization attempts
- Convergence metrics
- Time spent on each step
Enable detailed logging by modifying the print statements in the agent. The optimization history file contains detailed information about each step.
- Fork the repository
- Create a feature branch
- Make your changes
- Add tests if applicable
- Submit a pull request
This project is licensed under the MIT License.
For issues and questions:
- Check the optimization history file for detailed logs
- Verify your API keys are set correctly
- Ensure all dependencies are installed
- Check the HTML structure of your website
- Searcher: Finds similar websites for comparison
- Webpage Reader: Extracts LLM-readable content
- Reranker API: Scores content relevance
- Website Similarity Analyzer: Compares your website with competitors