Skip to content

feat(instrumentation): make the openai instrumentation context aware #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 15, 2024

Conversation

Dam-Buty
Copy link
Contributor

@Dam-Buty Dam-Buty commented Jul 11, 2024

The idea is to simplify the use of the openai instrumentation, making it more like the one in Python where you instrument the lib once, and each call is automatically captured and logged without having to instrument the result.

This is made possible by the AsyncLocalStorage context we have added in the last version.

Old syntax

    const response = await openai.chat.completions.create({
      model: 'gpt-3.5-turbo',
      messages: []
    });

    await client.instrumentation.openai(response);

    const run = thread.run({ name: "Bla" }).send()

    const response2 = await openai.chat.completions.create({
      model: 'gpt-3.5-turbo',
      messages: []
    });

    await client.instrumentation.openai(response2, run);

New syntax

client.instrumentation.openai();

// Will be logged as a simple generation outside of threads/steps
const response = await openai.chat.completions.create({
  model: 'gpt-3.5-turbo',
  messages: []
});

client.thread({ name: "Bla" }).wrap(async () => {
  // Will be logged as a step inside the "Bla" thread
  return openai.chat.completions.create({
      model: 'gpt-3.5-turbo',
      messages: []
  });
})

Copy link

linear bot commented Jul 11, 2024

ENG-1634 Change the OpenAI Instrumentation

It should be possible now with the context to instrument OpenAI call the same way we do in the Python SDK .

This means we just instrument the openai methods out of context, then when we intercept a call we determine where to put the generation :

  • if we have a step in the context we push it to that step
  • otherwise we just push the generation without a step

@Dam-Buty Dam-Buty requested a review from willydouhard July 15, 2024 07:38
@Dam-Buty Dam-Buty merged commit e5c0789 into main Jul 15, 2024
2 checks passed
@Dam-Buty Dam-Buty deleted the damien/eng-1634-change-the-openai-instrumentation branch August 1, 2024 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants