Insight-Agent 🧠

Your Multi-Source AI Research Partner

Overview

Insight-Agent is an intelligent research assistant that leverages multiple AI agents to provide comprehensive answers to complex research queries. The application uses a multi-agent approach with parallel processing capabilities for enhanced performance.

1. Researcher Agent: Analyzes the user's query, generates optimized search queries, and gathers information from multiple sources including Wikipedia and web pages via DuckDuckGo. Features parallel processing for concurrent web scraping and API calls.

2. Synthesizer Agent: Takes the raw data collected by the Researcher Agent and synthesizes it into a coherent, comprehensive answer tailored to the user's original query.

Features

• Smart Query Generation: Automatically generates effective search queries based on the user's request using Gemini 1.5 Flash
• Date-Aware Searches: Incorporates the current date to ensure time-sensitive queries return up-to-date information
• Multi-Source Research: Gathers information from Wikipedia and various web sources
• Parallel Processing: Concurrent web scraping and API calls for faster research results
• Content Synthesis: Combines information from multiple sources into a coherent, comprehensive response
• User-Friendly Interface: Simple Streamlit web interface for easy interaction

Technology Stack

• Frontend: Streamlit
• AI Models: Google's Gemini 1.5 Flash
• Web Scraping: BeautifulSoup4, Requests, aiohttp
• Search Tools: DuckDuckGo Search API
• Parallel Processing: asyncio, aiohttp, aiolimiter
• Security: certifi for SSL certificate verification

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/Insight-Agent.git
   cd Insight-Agent
   

2. Create and activate a virtual environment:

   python -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate
   

3. Install dependencies:

   pip install -r requirements.txt
   

4. Set up your Google API key:

   • Create a .streamlit/secrets.toml file
   • Add your Google API key: GOOGLE_API_KEY = "your-api-key-here"

Usage

1. Start the application:

   streamlit run app.py
   

2. Enter your research query in the text input field

3. Click "Launch Agents" to start the research process

4. The Researcher Agent will concurrently:

   • Generate optimized search queries
   • Search Wikipedia
   • Fetch and process multiple web pages simultaneously

5. The Synthesizer Agent will process all gathered information

6. Review the comprehensive answer provided

Performance Features

Parallel Processing

The application implements parallel processing in the Researcher Agent using:

• asyncio: For managing concurrent tasks
• aiohttp: For non-blocking HTTP requests
• aiolimiter: For rate limiting to prevent server overload
• SSL Certificate Handling: Using certifi for secure connections

This implementation allows:

• Concurrent web page scraping
• Simultaneous API calls
• Faster research results
• Responsible web scraping with rate limiting

Project Structure

Insight-Agent/
├── .streamlit/
│   └── secrets.toml       # Configuration file for API keys
├── agents/
│   ├── researcher.py      # Researcher agent with parallel processing
│   └── synthesizer.py     # Synthesizer agent for analysis
├── app.py                 # Main Streamlit application
├── requirements.txt       # Project dependencies
└── README.md             # This file

Dependencies

• streamlit==1.35.0
• google-generativeai==0.7.1
• requests==2.32.3
• beautifulsoup4==4.12.3
• googlesearch-python==1.2.3
• duckduckgo_search
• aiohttp==3.9.1
• asyncio==3.4.3
• aiolimiter==1.1.0
• certifi==2024.2.2

License

This project is licensed under the MIT License. See the LICENSE file for more details.

Contributing

We welcome contributions to improve Insight-Agent! Here's how you can contribute:

Getting Started

1. Fork the repository
2. Create a new branch: git checkout -b feature/your-feature-name
3. Make your changes
4. Test your changes thoroughly
5. Commit your changes: git commit -m 'Add some feature'
6. Push to the branch: git push origin feature/your-feature-name
7. Submit a pull request

Contribution Guidelines

• Code Style: Follow the existing code style and formatting
• Documentation: Update documentation to reflect your changes
• Testing: Add tests for new features and ensure all tests pass
• Commit Messages: Write clear, concise commit messages describing your changes
• Pull Requests: Keep pull requests focused on a single feature or bug fix

Development Setup

1. Set up your development environment as described in the Installation section
2. Create a separate .streamlit/secrets.toml file for development with your API keys
3. Run tests before submitting your changes

Reporting Issues

If you find a bug or have a suggestion for improvement:

1. Check if the issue already exists in the Issues section
2. If not, create a new issue with a clear title and detailed description
3. Include steps to reproduce the bug or explain your feature request
4. Add relevant labels to the issue

Code of Conduct

• Be respectful and inclusive in your interactions
• Provide constructive feedback
• Focus on the best interests of the project
• Help create a positive community environment

Contact

👉 For your questions, you can reach me at bartugkaan.celebi@outlook.com