-
Notifications
You must be signed in to change notification settings - Fork 896
feat(exporter-otlp-*): support custom HTTP agents #5719
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat(exporter-otlp-*): support custom HTTP agents #5719
Conversation
af65d9f
to
7d4481b
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #5719 +/- ##
==========================================
- Coverage 95.04% 95.01% -0.04%
==========================================
Files 310 303 -7
Lines 8013 7964 -49
Branches 1620 1614 -6
==========================================
- Hits 7616 7567 -49
Misses 397 397
🚀 New features to boost your workflow:
|
@@ -31,4 +31,5 @@ export interface HttpRequestParameters { | |||
headers: () => Record<string, string>; | |||
compression: 'gzip' | 'none'; | |||
agentOptions: http.AgentOptions | https.AgentOptions; | |||
agent?: http.Agent | https.Agent; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there any way we could consolidate agentOptions
and agent
into a supplier format so one could do something like this:
// some options to shorten it that we provide
type HttpAgentFactory = (() => http.Agent | https.Agent));
function agentFromOptions(agentOptions: http.AgentOptions | https.AgentOptions): HttpAgentFactory {
// createAgent does the lazy-loading
return () => createAgent({agentOptions});
}
// then the user does
createHttpTransport({ agent: agentFromOptions({...})});
If we have createAgent()
lazy load http in the background, we could scrap agentOptions
on the HttpRequestParameters
.
Reason: The exporter interface already has a lot of options duplication. My goal with this interface here was that it would eventually be rolled into a new exporter creation interface that would get rid of that duplication (and a lot of complexity that follows from that).
If we could support all use-cases with one option down here, we'll also be able to do the same with a potential new exporter creation interface.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do like this a lot more yeah. We could also scrap the agent
property on the legacy options interface and make agentOptions: http.Agent | https.Agent | HttpAgentFactory
. We might also want to make HttpAgentFactory
return a promise to allow lazy loading the http
module easily. The factory approach would make this much less of a footgun in general really.
Updated to use a factory function for the agent, this iteration definitely feels cleaner than the previous one. I was also able to make what you attempted with #5220 work with the tests @pichlermarc (except now it breaks the instrumentation tests, looking into that) |
59b612a
to
06fa08b
Compare
@raphael-theriault-swi nice 🙂 Before doing another round of reviews: it looks like the API tests are now failing here due to the changed settings: opentelemetry-js/api/test/common/internal/version.test.ts Lines 27 to 28 in a4e9645
|
5ee1842
to
58f9655
Compare
// eslint-disable-next-line @typescript-eslint/no-require-imports | ||
} = require('./http-transport-utils'); | ||
// Lazy import to ensure that http/https is not required before instrumentations can wrap it. | ||
const imported = await import('./http-transport-utils.js'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By doing it this way, you can end up in a race condition where multiple _loadUtils
are running in parallel if it is called multiple times before the import gets resolved.
I suppose it is kind of hypothetical case, but in case you want to change this, you can take a look at #5746 where i've made a guard against this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wouldn't say this is just a hypothetical case since it's perfectly possible for two exporters for two different signals to start exporting concurrently. So this is a good point, but 1. in this cause it wouldn't be a problem because loading the transport utils has no side effects and 2. Node already has a module cache that does essentially the same thing.
@pichlermarc I'm looking into it, the failure is not reproducible on macOS (with Node 20.6.0) so it'll have to wait a bit until next week when I have time to set up an environment where it also fails |
this._parameters.url, | ||
this._parameters.agentOptions | ||
agent: await this._parameters.agent( | ||
new URL(this._parameters.url).protocol |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we add some exception handling here as new URL()
will throw an exception if input is invalid.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The URL is already validated by the time this is called so I didn't deem it necessary.
This PR is currently on hold as there's no way to make the tests work with |
7c9eae4
to
3166b11
Compare
@pichlermarc I've been banging my head for a week trying to migrate the tests to use something other than |
Which problem is this PR solving?
This makes it possible to use a custom
http.Agent
with the exporters, which enables some useful use cases, mainly proxying, without having to implement them in the SDK.Fixes #5711
Short description of the changes
A new
httpAgent
option was added, with a JSDoc comment explaining how it is a footgun andhttpAgentOptions
should be preferred whenever possible.Type of change
How Has This Been Tested?
Added a test to make sure the custom agent is in fact being used.
Checklist: