Sequence Flow
- User utterance sent to Director Supervisor
- Predict next step: Director Supervisor reviews the current ‘world state’:
World state:
- Short term memory: Conversation history {userMessage1: "Hello", AIMessage1: "Hi, how can I help?"},
- Supervisors and Agents: {taskSupervisor: 'description: has a team of agents that execute tasks', financialAdvisorAgent: 'description: is an expert in helping the user with their finances'
- Previously completed and failed tasks
- Chain of thought & Tip prompting: We ask GPT4 to first reason about current progress and suggest the next task. Hilariously, providing a tip has been found to increase performance.
- Select Supervisor: After Director Supervisor predicts the next step, it is given the availability of other Supervisor agents it can use. It picks the most suitable supervisor to handle the next step.
- Summarise conversation history based on step 1 prediction and the supervisor director has picked, the complete chat history is "contextualised" - only the most relevant messages are used for the final evaluation step
- Evaluate & deliver A final evaluation step is run to determine what agent function is executed to try and further achieve the user's goal.
Returning of task: Director is passed a summary of the task assigned by another supervisor and is told whether it is done successfully or not. If not completed successfully, a detailed review describes why not and what is needed to succeed next time.
- If the task is completed successfully: This pattern is looped until all tasks listed in the canvas are completed and the response is handed back to the user.
- If the task is not completed successfully: Director passes on the failed task and feedback to the Automatic curriculum agent, which suggests a new smaller, easier step to take to complete. This is passed on to the quartermaster, which analyses the error and the new step/task. It then creates new agents or tools to fulfil the task. This is passed back to the Director once complete and the director once again chooses an agent to complete the task, with that agent's new tool at its disposal.
This concludes the responsibilities of the Director Supervisor.
Additional Context We leverage GPT3.5 to self-ask questions to provide additional context and increase the chance of task success. Each question is paired with a concept that is used for retrieving the most relevant document from the knowledge graph. We feed the document content to GPT3.5 to answer the self-written questions.
Additional Context Prompt: You are a helpful assistant that answers my question about {topic}. I will give you the following information: Question: ... You will answer the question based on the context ( only if available and helpful ) and your own knowledge of {topic} 1) Start your answer with " Answer: ". 2) Answer " Answer: Unknown " if you don’t know the answer .
Automic curriculum agent
Automatic curriculum prompt
You are a helpful assistant who tells me the next immediate task to do in {user request}. My ultimate goal is to discover as many diverse things as possible, accomplish as many diverse tasks as possible and become the best {user request category provider} in the world. I will give you the following information : {'Question 1: ...': 'Answer : ...', 'Question 2: ...': 'Answer : ...'} Tools available to Supervisor & agents: {tools}. Completed tasks so far: {completed tasks}. Failed tasks that are too hard: {failed tasks}. You must follow the following criteria: 1) You should act as a mentor and guide me to the next task based on my current learning progress. 2) Please be very specific about what further knowledge I need to get, how to complete a task or what tool to use. 3) The next task should follow a concise format, such as " Mine [quantity] [block]", "Craft[quantity][item]", "Smelt[quantity][item]", "Kill[quantity][mob]", "Cook[quantity][food]", "Equip[item]" etc. It should be a single phrase. Do not propose multiple tasks at the same time. Do not mention anything else. 4) The next task should not be too hard since I may not have the necessary resources or have learned enough skills to complete it yet. 5) The next task should be novel and interesting. I should look for rare resources, upgrade my equipment and tools using better materials, and discover new things. I should not be doing the same thing over and over again. 6) I may sometimes need to repeat some tasks if I need to collect more resources to complete more difficult tasks. Only repeat tasks if necessary. 7) Do not ask me to build or dig shelter even if it’s at night. I want to explore the world and discover new things. I don’t want to stay in one place. 8) Tasks that require information beyond the player ’ s status to verify should be avoided . For instance , " Placing 4 torches " and " Dig a 2x1x2 hole " are not ideal since they require visual confirmation from the screen . All the placing , building , planting , and trading tasks should be avoided . Do not propose task starting with these keywords . You should only respond in the format as described below : RESPONSE FORMAT : Reasoning: Based on the information I listed above , do reasoning about what the next task should be. Task: The next task. Here’s an example response: Reasoning: The inventory is empty now, chop down a tree to get some wood. Task: Obtain a wood log .
Evaluate and Execute: A final evaluation step is run to determine what agent function is executed to try and further achieve the user's goal.
Inspired by the delocalised processing of the human brain: this agent is a central coordinating agent, responsible for managing the overall process, including chat interactions, predicting the best next steps, and finding the best agent for executing specific tasks.
The process loops:
- Predict Next Step: For each iteration of the execution loop, DirectorAgent starts by making an informed prediction about what the best next step should be. The predictBestNextStep method will take three parameters:
- messages: An array of ChatMessage objects from working memory, representing the conversation history.
- previousAgent: An optional parameter representing the agent that was used in the previous step.
- terminationStr: An optional string used to signal the termination of the process.
-
Select Best Agent: Based on this prediction, DirectorAgent selects a best-fit agent persona.
-
Contextualize Chat History: Based on the prediction from step 1 and the agent persona in step 2, the complete chat history is "contextualized" and only the most relevant messages are used for the final evaluation step.
-
Evaluate and Execute: A final evaluation step is run to determine what agent function is executed to try and further achieve the user's goal.
The PlannerAgent, inspired by the prefrontal cortex is involved in planning and project management, with a focus on achieving set goals. It keeps checking that it's achieved the criteria for the goals set by the user.
The process loops:
- GoalRequired Is a goal required to be set to answer the question and if so
- GoalCheckDoes the user have clear goals that relate to the question in mind?
- ContextCheck Do I need any further context to answer this question?
- IfContextMissing Calls QuestionGeneratorAgent to find missing context and generate actions or questions needed
- IfNotContextMissing Either carries out the task with TaskAgent or Call WorkingMemoryAgent and pull appropriate guidance or information
Calls the NextBestQuestion function in Neo4j which infers on missing information needed from the graph and generates a list of questions to be answered. Pushes quantity to the front end and also tells the user how many questions need to be asked.
The WorkingMemory Agent Reads and Writes to the working memory:
- conversation context
- new web search information
- new user information
We take inspiration from how the brain stores events using 'Sparse Representation': Individual memories are represented in the brain by small sets of neurons, spread across, rather than in local areas in the hippocampus. In graph terms, this means only storing 'essential' connections and relationships, rather than trying to link every piece of information with everything else, which is what will happen in the working memory. Our ambition is to represent each key concept or entity in the graph with a limited number of highly relevant connections, avoiding excessive interlinking that could dilute the significance of each connection. How this functions exactly is yet to be worked out.
We're unsure whether few-shot will be enough to achieve 'cleaning up' the graph after a conversation has ended.
ProspectiveAgent: Reads and writes to prospective memory for appointments, dates or events that are due to happen in the future.
CsvAnalystAgent: This agent is designed to perform various operations on CSV data, such as adding columns, filtering rows, joining by column, ordering columns, sorting by column, summing up column values, and writing files.
DeveloperAgent: This agent seems to be focused on development-related tasks, including reading and writing files, and reading directories. It also has commented-out functions for running Pytest and initializing Poetry, indicating potential capabilities for Python development workflows.
ResearcherAgent: This agent is equipped for conducting web research, scraping text, and handling file operations (reading and writing files, reading directories).
SynthesizerAgent: The SynthesizerAgent is focused on synthesizing or combining information, supported by file reading and writing capabilities.
Coming soon
Coming Soon
Uses an inference engine (yet to be decided, could be as simple as a property check) that checks for the goal and decides what information is needed before being able to provide recommendations, options, advice or carry out a task.
Next Best Question Condenders: *Deepminds AMIE
When a new conversation starts, a copy of the episodic/semantic graph is made to store conversation, tasks etc. It's also used to infer and carry out most functions below. Once the conversation has finished, the conversation and any actions taken are summarised, and the graph is then restructured and formatted back into long-term memory to be more suitable and efficient (see long-term processing) for later use if new nodes and relationships have been made.
Stores: Workspace, Tasks, and current conversation chain.
Episodic memory is needed to recall past events – recent or distant. You use episodic memory when you remember personal experiences, such as what you had for lunch or when you attended a family gathering. Semantic Memory is needed to recall facts.
Used to remember appointments, dates or events that are due to happen in the future.
- Content
- ContentSource
- LastSourced
More to come!
