News Agents

The idea behind this project is to create a newsletter team that researches, creates, formats, and reviews content based on automated document scraping and formatting.

Features

• Create Agent Diagram for research
• Research
• Format Research
• Save Research
• Retrieve Research
• Perform Embeddings
• Create Agents to interact with research data
• Save research and retrieve data
• Create blacklist
• Proxy system
• Create a Telegram chatbot to receive selected articles

Diagram

graph TD
  A[Start]
  A --> B{Research Agents}
  B -->|Geopolitics| C[Geopolitics Agents]
  B -->|Economy| D[Economy Agents]
  B -->|International Relations| E[International Relations Agents]
  B -->|Cryptocurrencies| F[Cryptocurrencies Agents]
  B -->|Writing| G[Writing Agents]
  G --> H{Formatting Agents}
  H --> I{Review Agents}
  I -->|Done| J[End]
  C --> G
  D --> G
  E --> G
  F --> G

Loading

Exclusion List Websites

Some websites are ignored during news scraping due to the nature of their elements. They are:

• www.metropoles.com
• g1.globo.com
• dinheirama.com
• valor.globo.com
• www.terra.com.br
• www.viomundo.com.br
• sputniknewsbr.com.br
• www.bbc.com
• www.infomoney.com.br

Future Improvements

Using agents asynchronously to automate the collection of articles.

Used Stack

Back-end:

• crewai
• requests
• langchain_google_genai
• langchain_community
• python-dotenv
• langchain-chroma
• pyTelegramBotAPI
• "unstructured[md]"

Environment Variables

To run this project, you need to set the following environment variables in your .env file:

• NEWS_API_KEY
• GOOGLE_API_KEY

Sites to get the API keys:

• Gemini Pro
• text

Running Locally

Clone the repository:

git clone https://github.com/DevSilva-BR/Agents-newsletter.git

Navigate to the project directory:

cd my-project

Activate the virtual environment:

./Scripts/activate

Install dependencies:

pip install -r requirements.txt

Start the server:

python main.py