Skip to content

feat(exporter-otlp-*): support custom HTTP agents #5719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

raphael-theriault-swi
Copy link
Member

Which problem is this PR solving?

This makes it possible to use a custom http.Agent with the exporters, which enables some useful use cases, mainly proxying, without having to implement them in the SDK.

Fixes #5711

Short description of the changes

A new httpAgent option was added, with a JSDoc comment explaining how it is a footgun and httpAgentOptions should be preferred whenever possible.

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Added a test to make sure the custom agent is in fact being used.

Checklist:

  • Followed the style guidelines of this project
  • Unit tests have been added
  • Documentation has been updated

@raphael-theriault-swi raphael-theriault-swi requested a review from a team as a code owner May 29, 2025 20:01
Copy link

codecov bot commented May 29, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.01%. Comparing base (a4e9645) to head (30edb48).
Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5719      +/-   ##
==========================================
- Coverage   95.04%   95.01%   -0.04%     
==========================================
  Files         310      303       -7     
  Lines        8013     7964      -49     
  Branches     1620     1614       -6     
==========================================
- Hits         7616     7567      -49     
  Misses        397      397              
Files with missing lines Coverage Δ
.../configuration/convert-legacy-node-http-options.ts 100.00% <100.00%> (ø)
...ase/src/configuration/legacy-node-configuration.ts 100.00% <ø> (ø)
...-base/src/configuration/otlp-http-configuration.ts 100.00% <100.00%> (ø)
...packages/otlp-exporter-base/src/index-node-http.ts 100.00% <100.00%> (ø)
...rter-base/src/transport/http-exporter-transport.ts 94.73% <100.00%> (+0.29%) ⬆️
...xporter-base/src/transport/http-transport-utils.ts 100.00% <ø> (ø)

... and 10 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@@ -31,4 +31,5 @@ export interface HttpRequestParameters {
headers: () => Record<string, string>;
compression: 'gzip' | 'none';
agentOptions: http.AgentOptions | https.AgentOptions;
agent?: http.Agent | https.Agent;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any way we could consolidate agentOptions and agent into a supplier format so one could do something like this:

// some options to shorten it that we provide
type HttpAgentFactory = (() => http.Agent | https.Agent));
function agentFromOptions(agentOptions: http.AgentOptions | https.AgentOptions): HttpAgentFactory {
  // createAgent does the lazy-loading
  return () => createAgent({agentOptions});
}

// then the user does
createHttpTransport({ agent: agentFromOptions({...})});

If we have createAgent() lazy load http in the background, we could scrap agentOptions on the HttpRequestParameters.

Reason: The exporter interface already has a lot of options duplication. My goal with this interface here was that it would eventually be rolled into a new exporter creation interface that would get rid of that duplication (and a lot of complexity that follows from that).

If we could support all use-cases with one option down here, we'll also be able to do the same with a potential new exporter creation interface.

Copy link
Member Author

@raphael-theriault-swi raphael-theriault-swi Jun 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do like this a lot more yeah. We could also scrap the agent property on the legacy options interface and make agentOptions: http.Agent | https.Agent | HttpAgentFactory. We might also want to make HttpAgentFactory return a promise to allow lazy loading the http module easily. The factory approach would make this much less of a footgun in general really.

@raphael-theriault-swi
Copy link
Member Author

raphael-theriault-swi commented Jun 3, 2025

Updated to use a factory function for the agent, this iteration definitely feels cleaner than the previous one. I was also able to make what you attempted with #5220 work with the tests @pichlermarc (except now it breaks the instrumentation tests, looking into that)

@pichlermarc
Copy link
Member

@raphael-theriault-swi nice 🙂

Before doing another round of reviews: it looks like the API tests are now failing here due to the changed settings:

const pjson = require('../../../package.json');
assert.strictEqual(pjson.version, VERSION);
- could you have a look?

// eslint-disable-next-line @typescript-eslint/no-require-imports
} = require('./http-transport-utils');
// Lazy import to ensure that http/https is not required before instrumentations can wrap it.
const imported = await import('./http-transport-utils.js');

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By doing it this way, you can end up in a race condition where multiple _loadUtils are running in parallel if it is called multiple times before the import gets resolved.

I suppose it is kind of hypothetical case, but in case you want to change this, you can take a look at #5746 where i've made a guard against this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't say this is just a hypothetical case since it's perfectly possible for two exporters for two different signals to start exporting concurrently. So this is a good point, but 1. in this cause it wouldn't be a problem because loading the transport utils has no side effects and 2. Node already has a module cache that does essentially the same thing.

@raphael-theriault-swi
Copy link
Member Author

@pichlermarc I'm looking into it, the failure is not reproducible on macOS (with Node 20.6.0) so it'll have to wait a bit until next week when I have time to set up an environment where it also fails

this._parameters.url,
this._parameters.agentOptions
agent: await this._parameters.agent(
new URL(this._parameters.url).protocol
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add some exception handling here as new URL() will throw an exception if input is invalid.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The URL is already validated by the time this is called so I didn't deem it necessary.

@raphael-theriault-swi
Copy link
Member Author

This PR is currently on hold as there's no way to make the tests work with ts-node on all the Node.js versions we test in CI (and believe me I tried). I'm working on a new PR to move to tsx as discussed during the 2025/06/11 SIG.

@raphael-theriault-swi
Copy link
Member Author

@pichlermarc I've been banging my head for a week trying to migrate the tests to use something other than ts-node but it would require some pretty big changes since all of the module mocking with sinon kind of falls apart. I'll open an issue detailing my findings working on it in case anyone wants to pick up the work, but in the meantime I figured out how to make the tests work with this PR as-is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support passing a custom HTTP agent instance to HTTP exporters
4 participants