Updating naming and support for multiple completions

Author: djthorpe

README.md

@@ -120,24 +120,45 @@ func session(ctx context.Context, agent llm.Agent) error {
 You can add options to sessions, or to prompts. Different providers and models support
 different options.
 
+```go
+type Model interface {
+  // Set session-wide options
+  Context(...Opt) Context
+
+  // Add attachments (images, PDF's) to a user prompt
+  UserPrompt(string, ...Opt) Context
+
+  // Set embedding options
+  Embedding(context.Context, string, ...Opt) ([]float64, error)
+}
+
+type Context interface {
+  // Add single-use options when calling the model, which override
+  // session options. You can also attach files to a user prompt.
+  FromUser(context.Context, string, ...Opt) error
+}
+```
+
+The options are as follows:
+
 | Option | Ollama | Anthropic | Mistral | OpenAI | Description |
 |--------|--------|-----------|---------|--------|-------------|
 | `llm.WithTemperature(float64)` | Yes | Yes | Yes | - | What sampling temperature to use, between 0.0 and 1.0. Higher values like 0.7 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. |
 | `llm.WithTopP(float64)` | Yes | Yes | Yes | - | Nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. |
 | `llm.WithTopK(uint64)` | Yes | Yes | No | - | Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative. |
 | `llm.WithMaxTokens(uint64)` | - | Yes | Yes | - | The maximum number of tokens to generate in the response. |
-| `llm.WithStream(func(llm.ContextContent))` | Can be enabled when tools are not used | Yes | Yes | - | Stream the response to a function. |
+| `llm.WithStream(func(llm.Completion))` | Can be enabled when tools are not used | Yes | Yes | - | Stream the response to a function. |
 | `llm.WithToolChoice(string, string, ...)` | No | Yes | Use `auto`, `any`, `none`, `required` or a function name. Only the first argument is used. | - | The tool to use for the model. |
 | `llm.WithToolKit(llm.ToolKit)` | Cannot be combined with streaming | Yes | Yes | - | The set of tools to use. |
 | `llm.WithStopSequence(string, string, ...)` | Yes | Yes | Yes | - | Stop generation if one of these tokens is detected. |
 | `llm.WithSystemPrompt(string)` | No | Yes | Yes | - | Set the system prompt for the model. |
 | `llm.WithSeed(uint64)` | No | Yes | Yes | - | The seed to use for random sampling. If set, different calls will generate deterministic results. |
 | `llm.WithFormat(string)` | No | Yes | Use `json_format` or `text` | - | The format of the response. For Mistral, you must also instruct the model to produce JSON yourself with a system or a user message. |
-| `mistral.WithPresencePenalty(float64)` | - | - | Yes | - | Determines how much the model penalizes the repetition of words or phrases. A higher presence penalty encourages the model to use a wider variety of words and phrases, making the output more diverse and creative. |
-| `mistral.WithFequencyPenalty(float64)` | - | - | Yes | - | Penalizes the repetition of words based on their frequency in the generated text. A higher frequency penalty discourages the model from repeating words that have already appeared frequently in the output, promoting diversity and reducing repetition. |
-| `mistral.WithPrediction(string)` | - | - | Yes | - | Enable users to specify expected results, optimizing response times by leveraging known or predictable content. This approach is especially effective for updating text documents or code files with minimal changes, reducing latency while maintaining high-quality results. |
-| `llm.WithSafePrompt()` | - | - | Yes | - | Whether to inject a safety prompt before all conversations. |
-| `llm.WithNumCompletions(uint64)` | - | - | Yes | - | Number of completions to return for each request. |
+| `mistral.WithPresencePenalty(float64)` | No | No | Yes | - | Determines how much the model penalizes the repetition of words or phrases. A higher presence penalty encourages the model to use a wider variety of words and phrases, making the output more diverse and creative. |
+| `mistral.WithFequencyPenalty(float64)` | No | No | Yes | - | Penalizes the repetition of words based on their frequency in the generated text. A higher frequency penalty discourages the model from repeating words that have already appeared frequently in the output, promoting diversity and reducing repetition. |
+| `mistral.WithPrediction(string)` | No | No | Yes | - | Enable users to specify expected results, optimizing response times by leveraging known or predictable content. This approach is especially effective for updating text documents or code files with minimal changes, reducing latency while maintaining high-quality results. |
+| `llm.WithSafePrompt()` | No | No | Yes | - | Whether to inject a safety prompt before all conversations. |
+| `llm.WithNumCompletions(uint64)` | No | No | Yes | - | Number of completions to return for each request. |
 | `llm.WithAttachment(io.Reader)` | Yes | Yes | Yes | - | Attach a file to a user prompt. It is the responsibility of the caller to close the reader. |
 | `antropic.WithEphemeral()` | No | Yes | No | - | Attachments should be cached server-side |
 | `antropic.WithCitations()` | No | Yes | No | - | Attachments should be used in citations |

context.go

@@ -5,21 +5,28 @@ import "context"
 //////////////////////////////////////////////////////////////////
 // TYPES
 
-// ContextContent is the content of the last context message
-type ContextContent interface {
+// Completion is the content of the last context message
+type Completion interface {
+	// Return the number of completions, which is ususally 1 unless
+	// WithNumCompletions was used when calling the model
+	Num() int
+
 	// Return the current session role, which can be system, assistant, user, tool, tool_result, ...
+	// If this is a completion, the role is usually 'assistant'
 	Role() string
 
-	// Return the current session text, or empty string if no text was returned
-	Text() string
+	// Return the text for the last completion. If multiple completions are not
+	// supported, the argument is ignored.
+	Text(int) string
 
-	// Return the current session tool calls, or empty if no tool calls were made
-	ToolCalls() []ToolCall
+	// Return the current session tool calls given the completion index.
+	// Will return nil if no tool calls were returned
+	ToolCalls(int) []ToolCall
 }
 
 // Context is fed to the agent to generate a response
 type Context interface {
-	ContextContent
+	Completion
 
 	// Generate a response from a user prompt (with attachments and
 	// other options)