Skip to content

Commit dc26227

Browse files
GyarbijGyarbij
authored
Merge pull request #14 from Gyarbij/dev
1.0.3
2 parents 579d2da + 0ff5b48 commit dc26227

File tree

3 files changed

+347
-298
lines changed

3 files changed

+347
-298
lines changed

‎README.md

Lines changed: 59 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -79,14 +79,16 @@ docker run -d -p 11437:11437 --name=azure-oai-proxy \
7979

8080
Environment Variables
8181

82-
| Parameters | Description | Default Value |
83-
| :------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------- |
84-
| AZURE_OPENAI_PROXY_ADDRESS | Service listening address | 0.0.0.0:11437 |
85-
| AZURE_OPENAI_PROXY_MODE | Proxy mode, can be either "azure" or "openai". | azure |
86-
| AZURE_OPENAI_ENDPOINT | Azure OpenAI Endpoint, usually looks like https://{custom}.openai.azure.com. Required. | |
87-
| AZURE_OPENAI_APIVERSION | Azure OpenAI API version. Default is 2024-05-01-preview. | 2024-05-01-preview |
88-
| AZURE_OPENAI_MODEL_MAPPER (DEPRECATED) | A comma-separated list of model=deployment pairs. Maps model names to deployment names. For example, `gpt-3.5-turbo=gpt-35-turbo`, `gpt-3.5-turbo-0301=gpt-35-turbo-0301`. If there is no match, the proxy will pass model as deployment name directly (in fact, most Azure model names are same with OpenAI). | `gpt-3.5-turbo=gpt-35-turbo`<br/>`gpt-3.5-turbo-0301=gpt-35-turbo-0301` |
89-
| AZURE_OPENAI_TOKEN | Azure OpenAI API Token. If this environment variable is set, the token in the request header will be ignored. | "" |
82+
Here's the updated markdown table including a column for required:
83+
84+
| Parameters | Description | Default Value | Required |
85+
| :------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------- | :------- |
86+
| AZURE_OPENAI_PROXY_ADDRESS | Service listening address | 0.0.0.0:11437 | No |
87+
| AZURE_OPENAI_PROXY_MODE | Proxy mode, can be either "azure" or "openai". | azure | No |
88+
| AZURE_OPENAI_ENDPOINT | Azure OpenAI Endpoint, usually looks like https://{YOURDEPLOYMENT}.openai.azure.com. | | Yes |
89+
| AZURE_OPENAI_APIVERSION | Azure OpenAI API version. Default is 2024-05-01-preview. | 2024-05-01-preview | No |
90+
| AZURE_OPENAI_MODEL_MAPPER (Use for custom deployment names) | A comma-separated list of model=deployment pairs. Maps model names to deployment names. For example, `gpt-3.5-turbo=gpt-35-turbo`, `gpt-3.5-turbo-0301=gpt-35-turbo-0301`. If there is no match, the proxy will pass model as deployment name directly (most Azure model names are the same as OpenAI). | "" | No |
91+
| AZURE_OPENAI_TOKEN | Azure OpenAI API Token. If this environment variable is set, the token in the request header will be ignored. | "" | No |
9092

9193
Use in command line
9294

@@ -126,14 +128,23 @@ export HTTPS_PROXY=https://{your-domain}.com
126128

127129
## Deploy
128130

129-
Deploying through Docker
131+
Docker Normal Deployment
130132

131133
```shell
132134
docker pull gyarbij/azure-oai-proxy:latest
133135
docker run -p 11437:11437 --name=azure-oai-proxy \
134136
--env AZURE_OPENAI_ENDPOINT=https://{YOURENDPOINT}.openai.azure.com/ \
135137
gyarbij/azure-oai-proxy:latest
136138
```
139+
Docker with custom deployment names
140+
141+
```shell
142+
docker pull gyarbij/azure-oai-proxy:latest
143+
docker run -p 11437:11437 --name=azure-oai-proxy \
144+
--env AZURE_OPENAI_ENDPOINT=https://{YOURENDPOINT}.openai.azure.com/ \
145+
--env AZURE_OPENAI_MODEL_MAPPER=gpt-3.5-turbo=dev-g35-turbo,gpt-4=gpt-4ooo \
146+
gyarbij/azure-oai-proxy:latest
147+
```
137148

138149
Calling
139150

@@ -147,6 +158,45 @@ curl https://localhost:11437/v1/chat/completions \
147158
}'
148159
```
149160

161+
## Model Mapping Mechanism (Used for Custom deployment names)
162+
163+
These are the default mappings for the most common models, if your Azure OpenAI deployment uses different names, you can set the `AZURE_OPENAI_MODEL_MAPPER` environment variable to define custom mappings.:
164+
165+
| OpenAI Model | Azure OpenAI Model |
166+
|---------------------------------|-------------------------------|
167+
| `"gpt-3.5-turbo"` | `"gpt-35-turbo"` |
168+
| `"gpt-3.5-turbo-0125"` | `"gpt-35-turbo-0125"` |
169+
| `"gpt-3.5-turbo-0613"` | `"gpt-35-turbo-0613"` |
170+
| `"gpt-3.5-turbo-1106"` | `"gpt-35-turbo-1106"` |
171+
| `"gpt-3.5-turbo-16k-0613"` | `"gpt-35-turbo-16k-0613"` |
172+
| `"gpt-3.5-turbo-instruct-0914"` | `"gpt-35-turbo-instruct-0914"`|
173+
| `"gpt-4"` | `"gpt-4-0613"` |
174+
| `"gpt-4-32k"` | `"gpt-4-32k"` |
175+
| `"gpt-4-32k-0613"` | `"gpt-4-32k-0613"` |
176+
| `"gpt-4o"` | `"gpt-4o"` |
177+
| `"gpt-4o-2024-05-13"` | `"gpt-4o-2024-05-13"` |
178+
| `"gpt-4-turbo"` | `"gpt-4-turbo"` |
179+
| `"gpt-4-vision-preview"` | `"gpt-4-vision-preview"` |
180+
| `"gpt-4-turbo-2024-04-09"` | `"gpt-4-turbo-2024-04-09"` |
181+
| `"gpt-4-1106-preview"` | `"gpt-4-1106-preview"` |
182+
| `"text-embedding-ada-002"` | `"text-embedding-ada-002"` |
183+
| `"dall-e-2"` | `"dall-e-2"` |
184+
| `"dall-e-3"` | `"dall-e-3"` |
185+
| `"babbage-002"` | `"babbage-002"` |
186+
| `"davinci-002"` | `"davinci-002"` |
187+
| `"whisper-1"` | `"whisper"` |
188+
| `"tts-1"` | `"tts"` |
189+
| `"tts-1-hd"` | `"tts-hd"` |
190+
| `"text-embedding-3-small"` | `"text-embedding-3-small-1"` |
191+
| `"text-embedding-3-large"` | `"text-embedding-3-large-1"` |
192+
193+
For custom fine-tuned models, the model name can be passed directly. For models with deployment names different from the model names, custom mapping relationships can be defined, such as:
194+
195+
| Model Name | Deployment Name |
196+
| :----------------- | :--------------------------- |
197+
| gpt-3.5-turbo | gpt-35-turbo-upgrade |
198+
| gpt-3.5-turbo-0301 | gpt-35-turbo-0301-fine-tuned |
199+
150200
## Recently Updated
151201

152202
+ 2024-06-23 Implemented dynamic model fetching for `/v1/models endpoint`, replacing hardcoded model list.
@@ -163,21 +213,6 @@ curl https://localhost:11437/v1/chat/completions \
163213
+ 2024-06-22 Updated model mappings to include the latest models (gpt-4-turbo, gpt-4-vision-preview, dall-e-3).
164214
+ 2024-06-23 Added support for deployments management (/deployments).
165215

166-
## Model Mapping Mechanism (DEPRECATED)
167-
168-
There are a series of rules for model mapping pre-defined in `AZURE_OPENAI_MODEL_MAPPER`, and the default configuration basically satisfies the mapping of all Azure models. The rules include:
169-
170-
- `gpt-3.5-turbo` -> `gpt-35-turbo`
171-
- `gpt-3.5-turbo-0301` -> `gpt-35-turbo-0301`
172-
- A mapping mechanism that pass model name directly as fallback.
173-
174-
For custom fine-tuned models, the model name can be passed directly. For models with deployment names different from the model names, custom mapping relationships can be defined, such as:
175-
176-
| Model Name | Deployment Name |
177-
| :----------------- | :--------------------------- |
178-
| gpt-3.5-turbo | gpt-35-turbo-upgrade |
179-
| gpt-3.5-turbo-0301 | gpt-35-turbo-0301-fine-tuned |
180-
181216
## Contributing
182217

183218
We welcome contributions! Rest TBD.

0 commit comments

Comments
 (0)