| title | emoji | colorFrom | colorTo | sdk | sdk_version | app_file | pinned | hf_oauth | hf_oauth_expiration_minutes |
|---|---|---|---|---|---|---|---|---|---|
Agent GAIA |
🏆 |
pink |
indigo |
gradio |
5.33.0 |
app.py |
false |
true |
480 |
This project is an AI agent built for the GAIA benchmark as part of the Hugging Face Agents course. It combines different LLM models and multimodal tools to reason over text, audio, images and video to solve complex tasks.
The agent includes a variety of tools for handling diverse input types:
- Vision Tool: Analyze images using Gemini Vision.
- YouTube Frame Extractor: Sample video frames from YouTube at regular intervals.
- YouTube QA Tool: Ask questions about video content using Gemini via file URI.
- OCR Tool: Extract text from images using Tesseract.
- Audio Transcriber: Transcribe audio files and YouTube videos using Whisper.
- File Tools: Read plain text, download files from URLs, and summarize CSV or Excel files.
These tools are defined using the @tool decorator from the smolagents library, making them callable by the agent during task execution.
Gemini 2.5 Flash(via Google's Generative AI API)- Whisper for speech-to-text transcription
- Hugging Face Transformers (optional local model support)
- LiteLLM as a unified interface for calling external language models
- Install all required dependencies using
pip install -r requirements.txt- Convfigure environment with API_KEYS
echo "GEMINI_API_KEY=your_key_here" > .env
echo "HF_TOKEN=your_hf_token" >> .env- Run the app
python app.py