Skip to content

Commit 48237b2

Browse files
Deep research example
1 parent f0e1691 commit 48237b2

File tree

13 files changed

+554
-3
lines changed

13 files changed

+554
-3
lines changed
Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
# Deep Researcher Agent
2+
3+
![Demo](./assets/demo.png)
4+
5+
A multi-stage AI-powered research workflow agent that automates comprehensive web research, analysis, and report generation using Agno, Scrapegraph, and Nebius AI.
6+
7+
8+
## Features
9+
10+
- **Multi-Stage Research Workflow**: Automated pipeline for searching, analyzing, and reporting
11+
- **Web Scraping**: Advanced data extraction with Scrapegraph
12+
- **AI-Powered Analysis**: Uses Nebius AI for intelligent synthesis
13+
- **Streamlit Web UI**: Modern, interactive interface
14+
- **MCP Server**: Model Context Protocol server for integration
15+
- **Command-Line Support**: Run research tasks directly from the terminal
16+
17+
## How It Works
18+
19+
![Workflow](./assets/workflow.gif)
20+
21+
1. **Searcher**: Finds and extracts high-quality, up-to-date information from the web using Scrapegraph and Nebius AI.
22+
2. **Analyst**: Synthesizes, interprets, and organizes the research findings, highlighting key insights and trends.
23+
3. **Writer**: Crafts a clear, structured, and actionable report, including references and recommendations.
24+
25+
> **Workflow:**
26+
>
27+
> - Input a research topic or question
28+
> - The agent orchestrates web search, analysis, and report writing in sequence
29+
> - Results are presented in a user-friendly format (web or CLI)
30+
31+
32+
## Prerequisites
33+
34+
- Python 3.10+
35+
- [uv](https://github.com/astral-sh/uv) for dependency management
36+
- API keys for [Nebius AI](https://dub.sh/nebius) and [Scrapegraph](https://dub.sh/scrapegraphai)
37+
38+
39+
40+
## Installation
41+
42+
Follow these steps to set up the **Deep Researcher Agent** on your machine:
43+
44+
1. **Install `uv`** (if you don’t have it):
45+
46+
```bash
47+
curl -LsSf https://astral.sh/uv/install.sh | sh
48+
```
49+
50+
2. **Clone the repository:**
51+
52+
```bash
53+
git clone https://github.com/Arindam200/awesome-ai-apps.git
54+
```
55+
56+
3. **Navigate to the Deep Researcher Agent directory:**
57+
58+
```bash
59+
cd awesome-ai-apps/advance_ai_agents/deep_researcher_agent
60+
```
61+
62+
4. **Install all dependencies:**
63+
64+
```bash
65+
uv sync
66+
```
67+
68+
## Environment Setup
69+
70+
Create a `.env` file in the project root with your API keys:
71+
72+
```env
73+
NEBIUS_API_KEY=your_nebius_api_key_here
74+
SGAI_API_KEY=your_scrapegraph_api_key_here
75+
```
76+
77+
78+
## Usage
79+
80+
![usage](./assets/usage.gif)
81+
82+
You can use the Deep Researcher Agent in three ways. Each method below includes a demo image so you know what to expect.
83+
84+
### Web Interface
85+
86+
Run the Streamlit app:
87+
88+
```bash
89+
uv run streamlit run app.py
90+
```
91+
92+
Open your browser at [http://localhost:8501](http://localhost:8501)
93+
94+
What it looks like:
95+
96+
![demo](./assets/demo.png)
97+
98+
### Command Line
99+
100+
Run research directly from the command line:
101+
102+
```bash
103+
uv run python agents.py
104+
```
105+
106+
What it looks like:
107+
108+
![Terminal Demo](./assets/terminal-demo.png)
109+
110+
### MCP Server
111+
112+
Add the following configuration to your .cursor/mcp.json or Claude/claude_desktop_config.json file (adjust paths and API keys as needed):
113+
114+
```json
115+
{
116+
"mcpServers": {
117+
"deep_researcher_agent": {
118+
"command": "python",
119+
"args": [
120+
"--directory",
121+
"/Your/Path/to/directory/awesome-ai-apps/advance_ai_agents/deep_researcher_agent",
122+
"run",
123+
"server.py"
124+
],
125+
"env": {
126+
"NEBIUS_API_KEY": "your_nebius_api_key_here",
127+
"SGAI_API_KEY": "your_scrapegraph_api_key_here"
128+
}
129+
}
130+
}
131+
}
132+
```
133+
134+
This allows tools like Claude Desktop to manage and launch the MCP server automatically.
135+
136+
![Claude Desktop Demo](./assets/mcp-demo.png)
137+
138+
139+
140+
## Project Structure
141+
142+
```
143+
deep_researcher_agent/
144+
├── app.py # Streamlit web interface
145+
├── agents.py # Core agent workflow
146+
├── server.py # MCP server
147+
├── assets/ # Static assets (images)
148+
├── pyproject.toml # Project configuration
149+
└── README.md # This file
150+
```
151+
152+
---
153+
154+
## Development
155+
156+
### Code Formatting
157+
158+
```bash
159+
uv run black .
160+
uv run isort .
161+
```
162+
163+
### Type Checking
164+
165+
```bash
166+
uv run mypy .
167+
```
168+
169+
### Testing
170+
171+
```bash
172+
uv run pytest
173+
```
174+
175+
---
176+
177+
## Contributing
178+
179+
Contributions are welcome! Please feel free to submit a Pull Request or open an issue.
180+
181+
---
182+
183+
## Acknowledgments
184+
185+
- [Agno](https://www.agno.com/) for agent orchestration
186+
- [Scrapegraph](https://dub.sh/scrapegraphai) for web scraping
187+
- [Nebius AI Studio](https://studio.nebius.com/) for AI model access
188+
- [Streamlit](https://streamlit.io/) for the web interface
189+
190+
191+
## Author
192+
193+
Developed with ❤️ by [Arindam Majumder](https://www.youtube.com/c/Arindam_1729)
Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
import os
2+
from typing import Iterator
3+
4+
from agno.agent import Agent
5+
from agno.models.nebius import Nebius
6+
7+
# from agno.tools.scrapegraph import ScrapeGraphTools
8+
from agno.tools.duckduckgo import DuckDuckGoTools
9+
from agno.utils.log import logger
10+
from agno.workflow import RunResponse, Workflow
11+
12+
13+
class DeepResearcherAgent(Workflow):
14+
"""
15+
A multi-stage research workflow that:
16+
1. Gathers information from the web using advanced scraping tools.
17+
2. Analyzes and synthesizes the findings.
18+
3. Produces a clear, well-structured report.
19+
"""
20+
21+
searcher: Agent = Agent(
22+
tools=[DuckDuckGoTools()],
23+
model=Nebius(
24+
id="deepseek-ai/DeepSeek-V3-0324", api_key=os.getenv("NEBIUS_API_KEY")
25+
),
26+
show_tool_calls=True,
27+
markdown=True,
28+
description=(
29+
"You are ResearchBot-X, an expert at finding and extracting high-quality, "
30+
"up-to-date information from the web. Your job is to gather comprehensive, "
31+
"reliable, and diverse sources on the given topic."
32+
),
33+
instructions=(
34+
"1. Search for the most recent and authoritative and up-to-date sources (news, blogs, official docs, research papers, forums, etc.) on the topic.\n"
35+
"2. Extract key facts, statistics, and expert opinions.\n"
36+
"3. Cover multiple perspectives and highlight any disagreements or controversies.\n"
37+
"4. Include relevant statistics, data, and expert opinions where possible.\n"
38+
"5. Organize your findings in a clear, structured format (e.g., markdown table or sections by source type).\n"
39+
"6. If the topic is ambiguous, clarify with the user before proceeding.\n"
40+
"7. Be as comprehensive and verbose as possible—err on the side of including more detail.\n"
41+
"8. Mention the References & Sources of the Content. (It's Must)"
42+
),
43+
)
44+
45+
# Analyst: Synthesizes and interprets the research findings
46+
analyst: Agent = Agent(
47+
model=Nebius(
48+
id="deepseek-ai/DeepSeek-V3-0324", api_key=os.getenv("NEBIUS_API_KEY")
49+
),
50+
markdown=True,
51+
description=(
52+
"You are AnalystBot-X, a critical thinker who synthesizes research findings "
53+
"into actionable insights. Your job is to analyze, compare, and interpret the "
54+
"information provided by the researcher."
55+
),
56+
instructions=(
57+
"1. Identify key themes, trends, and contradictions in the research.\n"
58+
"2. Highlight the most important findings and their implications.\n"
59+
"3. Suggest areas for further investigation if gaps are found.\n"
60+
"4. Present your analysis in a structured, easy-to-read format.\n"
61+
"5. Extract and list ONLY the reference links or sources that were ACTUALLY found and provided by the researcher in their findings. Do NOT create, invent, or hallucinate any links.\n"
62+
"6. If no links were provided by the researcher, do not include a References section.\n"
63+
"7. Don't add hallucinations or make up information. Use ONLY the links that were explicitly passed to you by the researcher.\n"
64+
"8. Verify that each link you include was actually present in the researcher's findings before listing it.\n"
65+
"9. If there's no Link found from the previous agent then just say, No reference Found."
66+
),
67+
)
68+
69+
# Writer: Produces a final, polished report
70+
writer: Agent = Agent(
71+
model=Nebius(
72+
id="deepseek-ai/DeepSeek-V3-0324", api_key=os.getenv("NEBIUS_API_KEY")
73+
),
74+
markdown=True,
75+
description=(
76+
"You are WriterBot-X, a professional technical writer. Your job is to craft "
77+
"a clear, engaging, and well-structured report based on the analyst's summary."
78+
),
79+
instructions=(
80+
"1. Write an engaging introduction that sets the context.\n"
81+
"2. Organize the main findings into logical sections with headings.\n"
82+
"3. Use bullet points, tables, or lists for clarity where appropriate.\n"
83+
"4. Conclude with a summary and actionable recommendations.\n"
84+
"5. Include a References & Sources section ONLY if the analyst provided actual links from their analysis.\n"
85+
"6. Use ONLY the reference links that were explicitly provided by the analyst in their analysis. Do NOT create, invent, or hallucinate any links.\n"
86+
"7. If the analyst provided links, format them as clickable markdown links in the References section.\n"
87+
"8. If no links were provided by the analyst, do not include a References section at all.\n"
88+
"9. Never add fake or made-up links - only use links that were actually found and passed through the research chain."
89+
),
90+
)
91+
92+
def run(self, topic: str) -> Iterator[RunResponse]:
93+
"""
94+
Orchestrates the research, analysis, and report writing process for a given topic.
95+
"""
96+
logger.info(f"Running deep researcher agent for topic: {topic}")
97+
98+
# Step 1: Research
99+
research_content = self.searcher.run(topic)
100+
# logger.info(f"Searcher content: {research_content.content}")
101+
102+
logger.info("Analysis started")
103+
# Step 2: Analysis
104+
analysis = self.analyst.run(research_content.content)
105+
# logger.info(f"Analyst analysis: {analysis.content}")
106+
107+
logger.info("Report Writing Started")
108+
# Step 3: Report Writing
109+
report = self.writer.run(analysis.content, stream=True)
110+
yield from report
111+
112+
113+
def run_research(query: str) -> str:
114+
agent = DeepResearcherAgent()
115+
final_report_iterator = agent.run(
116+
topic=query,
117+
)
118+
logger.info("Report Generated")
119+
120+
full_report = ""
121+
for chunk in final_report_iterator:
122+
if chunk.content:
123+
full_report += chunk.content
124+
125+
return full_report

0 commit comments

Comments
 (0)