Skip to content

Model context window is not respected #1177

@mrcfps

Description

@mrcfps

Describe the bug

The input tokens exceed the model context window size, causing execution errors.

Full error stack: Error: 400 This endpoint's maximum context length is 32768 tokens. However, you requested about 42832 tokens (42832 of text input). Please reduce the length of either one, or use the "middle-out" transform to compress your prompt automatically.
    at APIError.generate (/app/node_modules/.pnpm/openai@4.104.0_encoding@0.1.13_ws@8.18.3_zod@3.23.8/node_modules/openai/error.js:45:20)
    at OpenAI.makeStatusError (/app/node_modules/.pnpm/openai@4.104.0_encoding@0.1.13_ws@8.18.3_zod@3.23.8/node_modules/openai/core.js:302:33)
    at OpenAI.makeRequest (/app/node_modules/.pnpm/openai@4.104.0_encoding@0.1.13_ws@8.18.3_zod@3.23.8/node_modules/openai/core.js:346:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/node_modules/.pnpm/@langchain+openai@0.5.6_@langchain+core@0.3.29_openai@4.104.0_encoding@0.1.13_ws@8.18.3_zod@3_stakin3q32wwqgut65iyrtr74e/node_modules/@langchain/openai/dist/chat_models.cjs:2038:29
    at async RetryOperation._fn (/app/node_modules/.pnpm/p-retry@4.6.2/node_modules/p-retry/index.js:50:12)

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions