-
Notifications
You must be signed in to change notification settings - Fork 24
Product Requirements Document
The purpose of this document is to outline the product requirements for InferGPT, a personalized AI agent designed to understand and predict users' needs effectively, facilitating deep personal interactions, goal setting, task execution, and personalized recommendations.
InferGPT aims to be an AI that personalizes interactions based on the user's specific data, facilitating a seamless and intuitive experience. See repo readme and home wiki for more details.
and especially voyager paper for how InferGPT can learn new skills in unfamiliar domains.
Initially start as a Financial assistant to keep the scope narrow, but keep in mind our approach should then work for any direction the user wishes to send InferGPT and therefore be general.
User story map to be completed such as example
To be completed.
Get up to speed with current research and agent designs (in order of priority & insight):
More of the same & agent frameworks:
Recommend guides to get hands on & learn:
User Story: As a user, I want the AI to access my data from various sources so that it can understand me better.
Task: Enable InferGPT to pull data from local folders
Inspiration: AutoGen
Task: Create an onboarding conversation flow that gathers some high-level info about the user. e.g "What would you like me to know about you first, or I can help you set a goal and go from there?"
Task: Develop a secure API integration framework to pull data from designated data sources.
Task: Implement user permission protocols for data access.
User Story: As a user, I want to be able to easily from various sources so that they can understand me better. Task: Create front end
Inspiration:
- ChatGPT rip off
- Streamlit
- Gradio
Task: Front end should have the ability to type messages & select model usage.
Task: The front end should display what it's thinking, what phase of action it's at, and how many questions it needs to ask before it's confident in its response/task and source of information.
Inspiration::
Task: Create a feedback loop to refine AI's understanding based on user interactions.
User Story: As a user, I want the AI to perform complex tasks on my behalf and be personalised to my situation.
Task: Explore and decide on agent frameworks:
Inspiration
Task stream chain of thought to terminal
Task: Create Director Agent
Inspiration
Tools
- Automatic Curriculum
- Canvas
Task: Create Planner Agent
Tools
- Next best question
Task: Create Memory Agent
Inspiration Text to Cypher
User Story: As an AI, to be able to infer the next best question, personalise, and remember conversations and other functions, I'll need a memory.
Task: Create a knowledge graph that stores and processes personal user data. Task: Support text to cypher llm that can manage ontology, database and read/write to store. Task: Support goal setting with tracking and reminders. Task: Support next best question inference
Task: Create Data Agent
Task: Create Synthiser Agent
Task: Create Skill library & self verify AI User Story: As an AI, I need to be able to adapt to unseen situations. This will require creating a new custom agent, tool or skill to achieve my task. Voyager
Task: Create Automatic Curriculum AI User Story: As an AI, I need a progressive increase in difficulty to enable a manageable learning process, so I should provide myself with a steady stream of new tasks and challenges to achieve a given task or goal. Voyager
User Story: As a user, I want the AI to perform tasks on my behalf.
Task: Build a task management Canvas that the AI can use to execute user commands.
Inspiration:
- Evo Ninja
- AutoGen
- CrewAI
Task: Create a reporting system for task outcomes.
User Story: As a user, I want the AI to be very clear on my intent and if in doubt, ask questions to check understanding before responding with an answer or carrying out a task.
Task: Design an inference engine that utilizes the knowledge graph for deep understanding.
The AI should be able to:
- have a JSON list of agents that can be switched on and off (fed into director agent)
- clarify intent or task within a reasonable amount of questions (# to be defined - estimated 3 to 10)
- ingest data from approved sources with user consent.
- Maintain a conversational context and recall previous interactions.
- perform designated tasks as instructed by the user.
- create new agents and skills if tasked to action in an area it's unfamiliar with
- Use APIs or local LLMs
- The AI should be able to compete and be benched on public AI agent benchmarks
- AutoGPT Benchmark
- Agent Board
- Strech goal: reach #1 on the AutoGPT/GAIA benchmark!
- Outrageous goal: pass financial advisor exams?
Other benchmarks to consider: GAIA
- Data Privacy Laws and Regulations
- Third-Party Data Source APIs
- Open AI / Hardware / Cloud requirements for running LLMs
- Neo4j Desktop or cloud
- Any data source integrations without user consent
- Actions beyond the agreed-upon user permissions
- Data breaches and privacy concerns
- User reliance on AI for decision-making
- It becomes ASI /s
- Dynamic user interface:
- Local models / Ollama
- QLoRa and Self-learning LLMs when quartermaster creates new agent
- Expansion of data sources
- Predictive Needs / Forecast intent
- Recommend Actions
- Security: Ensure all data is encrypted and handled according to privacy laws.
- Scalability: Design the system to handle an increasing amount of user data and interactions.
- Performance: Optimize response times for real-time interaction.
- Self Rewarding Language Models
- Multi-agent performance AI Agents is all you need
Pointing to Michaels point, building a perfectly robust system is impossible. (using the "3 tescos example"). In that instance, companies need to change their mindset that it's ok for the AI to come back to the user with uncertainty. "Which one of these 3 transactions is it?" - as this is what a human would do. Instead at the moment, especially in software engineering, we say to ourselves: "it would be an error to return all 3, so we cant do it". Additionally, in human enviroments, new staff undergo training and are expected to make mistakes, if allow the same room for AI, only on the condition that it can improve itself, this will be the path to generally capable systems. Sure we do this today, but usually through extensive engineer and testing and once done, the AI isn't given scope to adapt hereon. (The reason for this is simple - systems haven't been capable of such a thing, even viably, until now!)
The system therefore needs the ability to fail, get given feedback, iterate and improve. To achieve this, the system will need:
- quartermaster
- access to its OWN code.
- long term & contextual memory