RAG with Llama3 on Groq

This cookbook shows how to do retrieval-augmented generation (RAG) using Llama3 on Groq.

For embeddings we can either use Ollama or OpenAI.

Note: Fork and clone this repository if needed

1. Create a virtual environment

python3 -m venv ~/.venvs/aienv
source ~/.venvs/aienv/bin/activate

2. Export your Groq API Key

export GROQ_API_KEY=***

3. Use Ollama or OpenAI for embeddings

Since Groq doesnt provide embeddings yet, you can either use Ollama or OpenAI for embeddings.

• To use Ollama for embeddings Install Ollama and run the nomic-embed-text model

ollama run nomic-embed-text

• To use OpenAI for embeddings, export your OpenAI API key

export OPENAI_API_KEY=sk-***

4. Install libraries

pip install -r cookbook/llms/groq/rag/requirements.txt

5. Run PgVector

Install docker desktop first.

• run using the docker run command

docker run -d \
  -e POSTGRES_DB=ai \
  -e POSTGRES_USER=ai \
  -e POSTGRES_PASSWORD=ai \
  -e PGDATA=/var/lib/postgresql/data/pgdata \
  -v pgvolume:/var/lib/postgresql/data \
  -p 5532:5432 \
  --name pgvector \
  phidata/pgvector:16

6. Run RAG App

streamlit run app.py

• Open localhost:8501 to view your RAG app.
• Add websites or PDFs and ask question.