Skip to content

Adding support to Azure API #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 78 commits into from
Jun 24, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
a32e681
Adding support to Azure API
Feb 9, 2024
ccd6961
merge main
ccreutzi May 23, 2024
26b1272
parameterize getApiKeyFromNvpOrEnv, allowing different env variables …
ccreutzi May 27, 2024
8bd236b
get basic Azure connection working
ccreutzi May 27, 2024
e009e86
even smaller timeout; failed to throw an error in GitHub once
ccreutzi May 27, 2024
a6b8d51
add ollamaChat class
ccreutzi May 27, 2024
4304636
CI setup for ollama
ccreutzi May 27, 2024
61a5152
typos
ccreutzi May 27, 2024
9914a55
disable verySmallTimeOutErrors test points, since they are flaky
ccreutzi May 27, 2024
cd9bbe2
Updated README.md for Azure and Ollama
ccreutzi May 28, 2024
8a2ea28
Remove GPT specific penalties from ollamaChat
ccreutzi May 29, 2024
dde7d95
Implement TopProbabilityNum and StopSequences for ollamaChat
ccreutzi Jun 4, 2024
8d351a2
increase default timeout to 120 seconds for ollamaChat
ccreutzi Jun 4, 2024
e229935
add TailFreeSampling_Z, add comment about currently unsupported ollam…
ccreutzi Jun 5, 2024
b67f5ef
new static method ollamaChat.models
ccreutzi Jun 5, 2024
bcf4d85
update API versions, following https://learn.microsoft.com/en-us/azur…
ccreutzi Jun 5, 2024
e51f5eb
typo in help header
ccreutzi Jun 5, 2024
08a8549
add azureChat and ollamaChat to functionSignatures.json
ccreutzi Jun 5, 2024
410a87b
Make StreamFun work with ollamaChat and azureChat
ccreutzi Jun 5, 2024
6b830a2
remove unused defaults, for more realistic coverage numbers
ccreutzi Jun 5, 2024
c349a58
Add test that azureChat with Seed fixes result
ccreutzi Jun 5, 2024
0815bdf
try telling codecov to not worry about test files
ccreutzi Jun 6, 2024
e8a900d
also ignore errorMessageCatalog.m in codecov, since almost all of it …
ccreutzi Jun 6, 2024
a2d65cd
ignore examples/data/* just like data/*
ccreutzi Jun 6, 2024
5851aca
Merge branch 'main' into AzureAPI
ccreutzi Jun 6, 2024
88f054b
remove disabled timeout tests
ccreutzi Jun 6, 2024
5ed246d
Add explanatory comment for missing key test.
ccreutzi Jun 6, 2024
3bd2208
changed wording as requested
ccreutzi Jun 6, 2024
beb41a5
simplify comment
ccreutzi Jun 6, 2024
8d50886
spell Ollama with a capital O
ccreutzi Jun 6, 2024
1a5ebb9
Fix capitalization: APIKey, by MathWorks naming standards.
ccreutzi Jun 6, 2024
72dca78
Codecov has problems with uploaded coverage. Try not using plus signs.
ccreutzi Jun 7, 2024
3639020
add unit tests for edge cases and errors in responseStreamer
ccreutzi Jun 7, 2024
a2b893a
Add test point for function calls
ccreutzi Jun 7, 2024
0dd0e91
CI setup stores the API key in a different variable
ccreutzi Jun 7, 2024
5d2fd99
for better coverage, run tools test through streaming API
ccreutzi Jun 7, 2024
4ae0248
Function calling on Azure.
ccreutzi Jun 7, 2024
735416b
Throw server errors as errors
ccreutzi Jun 10, 2024
95f2f13
For CI, add `$OPENAI_API_KEY` such that `openAIChat` works
ccreutzi Jun 10, 2024
492fefc
Short error messages for bad endpoints
ccreutzi Jun 11, 2024
3000dc4
Nicer help headers
ccreutzi Jun 11, 2024
0973f11
typos
ccreutzi Jun 12, 2024
238e11a
Rename openAIMessages to messageHistory
ccreutzi Jun 12, 2024
101cb59
Add openAIMessages fallback for backward compatibility
ccreutzi Jun 12, 2024
e095802
Modify chatbot example to use Ollama
ccreutzi Jun 12, 2024
f680fb5
Merge branch 'main' into AzureAPI
ccreutzi Jun 12, 2024
7e81d37
minimal and complete test for the backward compatibility function
ccreutzi Jun 12, 2024
2611327
Avoid bogus json
ccreutzi Jun 12, 2024
16a3833
Improve ollamaChat tab completion
ccreutzi Jun 12, 2024
b33dad5
Remove unused error ID
ccreutzi Jun 12, 2024
cdf0971
add test for Ollama chatbot example
ccreutzi Jun 13, 2024
3abfae4
mark trademarks
ccreutzi Jun 13, 2024
d577d36
Include ._* in .gitignore
ccreutzi Jun 13, 2024
466460d
Rename `TopProbabilityMass` → `TopP`, `TopProbabilityNum` → `TopK`
ccreutzi Jun 17, 2024
24059e9
Take properties out of `ollamaChat` that do not apply: Tools and API key
ccreutzi Jun 17, 2024
632ac22
Merge branch 'main' into AzureAPI
ccreutzi Jun 17, 2024
d5eb25d
Only drop `:latest` from model list
ccreutzi Jun 17, 2024
6108d0f
ollamaChat.models should never return <missing>
ccreutzi Jun 19, 2024
5200776
fix indentations changed by renaming `TopProbabilityMass` to `TopP`
ccreutzi Jun 19, 2024
32547fb
accept char and cellstr input for generate
ccreutzi Jun 19, 2024
4bd315c
split README.md by backend
ccreutzi Jun 19, 2024
fa9f06e
`FunctionNames` should only exist for connectors with tools
ccreutzi Jun 20, 2024
8336cbb
Fix link typo
ccreutzi Jun 20, 2024
0543e3e
Fix typo: This is not using OpenAI
ccreutzi Jun 20, 2024
3ebc529
update tests to expect correct errors
ccreutzi Jun 20, 2024
71f88a9
tabs to spaces
ccreutzi Jun 20, 2024
949fe42
`openAIImages` should derive from `needsAPIKey`
ccreutzi Jun 20, 2024
09b8662
test `NumCompletions`
ccreutzi Jun 20, 2024
a360f89
Ollama does not support `NumCompletions`
ccreutzi Jun 20, 2024
348fb67
`azureChat` should get endpoint and deployment from env
ccreutzi Jun 20, 2024
4fb866c
clean up comments
ccreutzi Jun 21, 2024
e29e65c
move error text to catalogue
ccreutzi Jun 21, 2024
b4eb44a
Reorder test points for maintainability
ccreutzi Jun 21, 2024
757f260
Add a streaming example for `ollamaChat`
ccreutzi Jun 21, 2024
56e6aaf
Log Ollama version during CI
ccreutzi Jun 24, 2024
f6a9106
Add Seed test to tollamaChat.m
ccreutzi Jun 24, 2024
392749f
Use message from error catalog
ccreutzi Jun 24, 2024
1ac24ff
Disable flaky test points
ccreutzi Jun 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion +llms/+internal/callOllamaChatAPI.m
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
nvp.StreamFun
end

URL = "http://localhost:11434/api/chat"; % TODO: model parameter
URL = "http://localhost:11434/api/chat";

% The JSON for StopSequences must have an array, and cannot say "stop": "foo".
% The easiest way to ensure that is to never pass in a scalar …
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about: "To ensure that the JSON for StopSequences has a non-scalar array (is a non-scalar array?), create a two-element array if a scalar is passed."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, in JSON, a one-element array is still an array and would be fine. MATLAB's jsonencode just doesn't create "stop": ["foo"] as we need it to. But we can make it produce "stop": ["foo","foo"] instead.

Expand Down
1 change: 1 addition & 0 deletions +llms/+utils/errorMessageCatalog.m
Original file line number Diff line number Diff line change
Expand Up @@ -58,4 +58,5 @@
catalog("llms:warningJsonInstruction") = "When using JSON mode, you must also prompt the model to produce JSON yourself via a system or user message.";
catalog("llms:apiReturnedError") = "Server error: ""{1}""";
catalog("llms:dimensionsMustBeSmallerThan") = "Dimensions must be less than or equal to {1}.";
catalog("llms:stream:responseStreamer:InvalidInput") = "Input does not have the expected json format, got ""{1}"".";
end
2 changes: 1 addition & 1 deletion doc/OpenAI.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# OpenAI™

Several function sin this repository connect MATLAB to the [OpenAI™ Chat Completions API](https://platform.openai.com/docs/guides/text-generation/chat-completions-api) (which powers ChatGPT™) and the [OpenAI Images API](https://platform.openai.com/docs/guides/images/image-generation-beta) (which powers DALL·E™).
Several functions in this repository connect MATLAB to the [OpenAI™ Chat Completions API](https://platform.openai.com/docs/guides/text-generation/chat-completions-api) (which powers ChatGPT™) and the [OpenAI Images API](https://platform.openai.com/docs/guides/images/image-generation-beta) (which powers DALL·E™).

To start using the OpenAI APIs, you first need to obtain OpenAI API keys. You are responsible for any fees OpenAI may charge for the use of their APIs. You should be familiar with the limitations and risks associated with using this technology, and you agree that you shall be solely responsible for full compliance with any terms that may apply to your use of the OpenAI APIs.

Expand Down
Binary file not shown.
5 changes: 0 additions & 5 deletions ollamaChat.m
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,6 @@
%
% SystemPrompt - System prompt.

% Ollama model properties not exposed:
% repeat_last_n, repeat_penalty - could not find an example where they made a difference
% mirostat, mirostat_eta, mirostat_tau - looking for the best API design
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we document this somewhere internal so we don't forget about it?



% Copyright 2024 The MathWorks, Inc.

properties
Expand Down
66 changes: 35 additions & 31 deletions tests/texampleTests.m
Original file line number Diff line number Diff line change
Expand Up @@ -35,43 +35,15 @@ function testAnalyzeScientificPapersUsingFunctionCalls(~)
AnalyzeScientificPapersUsingFunctionCalls;
end

function testProcessGeneratedTextinRealTimebyUsingChatGPTinStreamingMode(~)
ProcessGeneratedTextinRealTimebyUsingChatGPTinStreamingMode;
end

function testUsingDALLEToGenerateImages(~)
UsingDALLEToGenerateImages;
end

function testInformationRetrievalUsingOpenAIDocumentEmbedding(~)
InformationRetrievalUsingOpenAIDocumentEmbedding;
end

function testDescribeImagesUsingChatGPT(~)
DescribeImagesUsingChatGPT;
end

function testSummarizeLargeDocumentsUsingChatGPTandMATLAB(~)
SummarizeLargeDocumentsUsingChatGPTandMATLAB;
function testAnalyzeSentimentinTextUsingChatGPTinJSONMode(testCase)
testCase.verifyWarning(@AnalyzeSentimentinTextUsingChatGPTinJSONMode,...
"llms:warningJsonInstruction");
end

function testAnalyzeTextDataUsingParallelFunctionCallwithChatGPT(~)
AnalyzeTextDataUsingParallelFunctionCallwithChatGPT;
end

function testRetrievalAugmentedGenerationUsingChatGPTandMATLAB(~)
RetrievalAugmentedGenerationUsingChatGPTandMATLAB;
end

function testUsingDALLEToEditImages(~)
UsingDALLEToEditImages;
end

function testAnalyzeSentimentinTextUsingChatGPTinJSONMode(testCase)
testCase.verifyWarning(@AnalyzeSentimentinTextUsingChatGPTinJSONMode,...
"llms:warningJsonInstruction");
end

function testCreateSimpleChatBot(testCase,ChatBotExample)
% set up a fake input command, returning canned user prompts
count = 0;
Expand Down Expand Up @@ -112,6 +84,38 @@ function testCreateSimpleChatBot(testCase,ChatBotExample)
testCase.verifyEqual(count,find(prompts=="end",1));
testCase.verifySize(messages.Messages,[1 2*(count-1)]);
end

function testDescribeImagesUsingChatGPT(~)
DescribeImagesUsingChatGPT;
end

function testInformationRetrievalUsingOpenAIDocumentEmbedding(~)
InformationRetrievalUsingOpenAIDocumentEmbedding;
end

function testProcessGeneratedTextinRealTimebyUsingChatGPTinStreamingMode(~)
ProcessGeneratedTextinRealTimebyUsingChatGPTinStreamingMode;
end

function testProcessGeneratedTextInRealTimeByUsingOllamaInStreamingMode(~)
ProcessGeneratedTextInRealTimeByUsingOllamaInStreamingMode;
end

function testRetrievalAugmentedGenerationUsingChatGPTandMATLAB(~)
RetrievalAugmentedGenerationUsingChatGPTandMATLAB;
end

function testSummarizeLargeDocumentsUsingChatGPTandMATLAB(~)
SummarizeLargeDocumentsUsingChatGPTandMATLAB;
end

function testUsingDALLEToEditImages(~)
UsingDALLEToEditImages;
end

function testUsingDALLEToGenerateImages(~)
UsingDALLEToGenerateImages;
end
end

end
Expand Down
Loading