Struggling to run on Azure gpt-4o-mini

[habitoti]

I am really struggling getting paperless-ai up & running on an Azure gpt-40-mini deployment. Even with a simple one-pager I run into a 429/Rate Limit Exceeded error. The token rate limit is 1000 Token/min, which sure isn't a lot, but I wonder if anyone got such a setup running at all? How would I even increase the quota? I can go lower, but 1k is the max to configure seemingly for a standard deployment on the only available European server in Sweden.

I also just suppose I've put in the right endpoint, deployment name, API key and API version, since the configuration went through w/o complaining. If anyone successfully running such a setup could sample a valid configuration (and from where exactly to pick all that information in the Azure portal) -- that would be greatly appreciated!

This is the log for the simple one-pager:

[2025-03-25T17:00:01.000Z] [INFO] [DEBUG] Fetched page 1, got 0 tags. [DEBUG] Total so far: 0
[2025-03-25T17:00:01.005Z] [INFO] [DEBUG] Found own user ID: 3
[2025-03-25T17:00:01.010Z] [INFO] [DEBUG] Fetched page 1, got 1 documents. [DEBUG] Total so far: 1
[2025-03-25T17:00:01.111Z] [INFO] [DEBUG] Finished fetching. Found 1 documents.
[2025-03-25T17:00:01.162Z] [INFO] [DEBUG] Document 5 rights for AI User - processed
[2025-03-25T17:00:01.237Z] [INFO] [DEBUG] Thumbnail already cached
[2025-03-25T17:00:02.765Z] [ERROR] Failed to analyze document: RateLimitError: 429 Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2025-01-01-preview have exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 60 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
    at APIError.generate (/opt/paperless-ai/node_modules/openai/error.js:63:20)
    at AzureOpenAI.makeStatusError (/opt/paperless-ai/node_modules/openai/core.js:299:33)
    at AzureOpenAI.makeRequest (/opt/paperless-ai/node_modules/openai/core.js:343:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async AzureOpenAIService.analyzeDocument (/opt/paperless-ai/services/azureService.js:183:24)
    at async processDocument (/opt/paperless-ai/server.js:210:20)
    at async scanDocuments (/opt/paperless-ai/server.js:399:24)
    at async Task._execution (/opt/paperless-ai/server.js:543:9) {
  status: 429,
  headers: {
    'apim-request-id': 'fe1128f9-b0e9-4744-8d13-23cbf009a5b0',
    'content-length': '444',
    'content-type': 'application/json',
    date: 'Tue, 25 Mar 2025 17:00:02 GMT',
    'policy-id': 'DeploymentRatelimit-Token',
    'retry-after': '60',
    'strict-transport-security': 'max-age=31536000; includeSubDomains; preload',
    'x-content-type-options': 'nosniff',
    'x-ms-region': 'Sweden Central',
    'x-ratelimit-limit-requests': '10',
    'x-ratelimit-limit-tokens': '1000',
    'x-ratelimit-remaining-requests': '7',
    'x-ratelimit-reset-tokens': '60'
  },
  request_id: undefined,
  error: {
    code: '429',
    message: 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2025-01-01-preview have exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 60 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.'
  },
  code: '429',
  param: undefined,
  type: undefined
}
[2025-03-25T17:00:02.765Z] [INFO] Repsonse from AI service: {
  document: { tags: [], correspondent: null },
  metrics: null,
  error: '429 Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2025-01-01-preview have exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 60 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.'
}
[2025-03-25T17:00:02.766Z] [ERROR] [ERROR] processing document 5: Error: [ERROR] Document analysis failed: 429 Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2025-01-01-preview have exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 60 seconds. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit. For Free Account customers, upgrade to Pay as you Go here: https://aka.ms/429TrialUpgrade.
    at processDocument (/opt/paperless-ai/server.js:213:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async scanDocuments (/opt/paperless-ai/server.js:399:24)
    at async Task._execution (/opt/paperless-ai/server.js:543:9)
[2025-03-25T17:00:02.766Z] [INFO] [INFO] Task completed

[habitoti]

Never mind...upgraded from free tier to pay-as-you-go (actually thought with the initial $200 Budget it is not a free tier for at least the first month...). and now the rate limit is 8K tokens which seems sufficient and it works now.

[tzippy84]

Im struggling to find what to input for „Deployment“. Can you help me?

[habitoti]

I used "Standard" because I wanted to make sure to control the server location. I wanted to stay in Europe to leverage GDPR constraints. If you use anything "Global" you have no control about where your requests go.

[tzippy84]

Is „Standard“ what I have to put in the „Deployment“ field when configuring Azure for paperless-ai?
I know what to put in the fields endpoint, API key, API version but am struggling with „Deployment“.

[habitoti]

Ah, sorry...thought you were talking about the Azure setup...
In paperless-ai, you have to put the name of the model (as you named in in Azure) into the deployment field. In my case, that's "gpt-4o-mini".

[eav86]

Also got problems with the setup.

Ah, sorry...thought you were talking about the Azure setup... In paperless-ai, you have to put the name of the model (as you named in in Azure) into the deployment field. In my case, that's "gpt-4o-mini".

is it possible to release a documentation how to setup azure for it ? I just activated "Azure for Student" and tried to install it. But always getting error.

[Audiojoy72]

using this form of endpoint also does not work: https://xxxxxx.openai.azure.com/openai/deployments/gpt-4o/completions?api-version=2024-10-21
Used this version no also in der version field.
this seems to be the latest version to use. this 2025.xx.xx seems to be for audio/media

I read documentations and also asked gpt and this should be the right setting.
this endpoint and api also work in n8n.
i think there is maybe a restriction in azure foundry?!
maybe someone can help here

[habitoti]

This is my configuration, albeit for gpt-4o-mini:

Works w/o issues. Not sure whether this is also the valid API version for gpt-4o. At least you need to reduce the token size, because Azure restricts you to max. 8K (assuming you have a "standard" subscription...). For the free tier it's even just 1k, which is basically unusable for this usecase. But the subscription is very cheap for normal private usage, so no worries...

[Audiojoy72]

Thanx! That helped alot. It seem this does not work with gpt-4o or different token settings i could not figure out. with this setting above its working. its important to use the endpoint url in this way. https://.openai.azure.com does not work it must be: https://xxxxxx.openai.azure.com/openai/deployments/gpt-4o/completions?api-version i just see i made a typo with the version in the url and the version field. it seems it does not matter. keep the value in the Version field and just stop after "?api-version" The settings for other models may be different.

[habitoti]

I am just using https://paperless-ai12345678.openai.azure.com/ as endpoint, and this works for me. I think when using a full qualified URL like you do Paperless will still append the model and API version from below, and whether that works and which API version is ultimately used is rather random then and depends on Azures mood of the day. So I would opt for the simple URL and let Paperless construct the final one (as it works for me, it will also for you, I believe…)