Skip to content

[fix][client] prevent garbage collection of token-supplier #412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 11, 2025

Conversation

klaatu01
Copy link
Contributor

@klaatu01 klaatu01 commented May 6, 2025

When using the new async token‐supplier API (() => Promise<string>) introduced in #395, the JS Authentication object and its backing ThreadSafeFunction can be garbage‐collected if no live JS references remain. After the initial token expires, Pulsar’s broker will send a new auth challenge and the native code crashes when it tries to invoke a finalized TSFN.

By persisting a reference to the authentication object in the client it prevents the V8 garbage collector from dropping the token-supplier.

@klaatu01 klaatu01 changed the title fix: prevent garbage collection of Promise based Token Authentication fix: prevent garbage collection of async token‐supplier May 6, 2025
@klaatu01 klaatu01 changed the title fix: prevent garbage collection of async token‐supplier [fix][client] prevent garbage collection of token-supplier May 6, 2025
@klaatu01
Copy link
Contributor Author

klaatu01 commented May 7, 2025

@RobertIndie @BewareMyPower
Noticed that the tests failed because open handles prevented jest from exiting, I am assuming I need to make alterations to the close function to make sure we deference the authRef that the client is holding?

Sorry I haven't used C++ much before, am I right to assume that's what I need to do?

@BewareMyPower
Copy link
Contributor

Yeah, compared to a successful workflow before (https://github.com/apache/pulsar-client-node/actions/runs/13716725396/job/38363013887):

Jest has detected the following 1 open handle potentially keeping Jest from exiting:

  ●  Pulsar Logging

      54 |
      55 |   static setLogHandler(params) {
    > 56 |     PulsarBinding.Client.setLogHandler(params);
         |                          ^
      57 |   }
      58 |
      59 |   static genCertFile() {

      at Function.setLogHandler (src/Client.js:56:26)
      at Object.<anonymous> (tests/end_to_end.test.js:402:21)


> pulsar-client@1.14.0-rc.0 docs
> typedoc

It seems that there are two more open handles that keep Jest from exiting:

Token Supplier Callback

      22 | class AuthenticationToken {
      23 |   constructor(params) {
    > 24 |     this.binding = new PulsarBinding.Authentication('token', params);
         |                    ^
      25 |   }
      26 | }
      27 |

      at new AuthenticationToken (src/AuthenticationToken.js:24:20)
      at Object.<anonymous> (tests/end_to_end.test.js:1290:20)


  ●  Token Supplier Callback

      22 | class AuthenticationToken {
      23 |   constructor(params) {
    > 24 |     this.binding = new PulsarBinding.Authentication('token', params);
         |                    ^
      25 |   }
      26 | }
      27 |

      at new AuthenticationToken (src/AuthenticationToken.js:24:20)
      at Object.<anonymous> (tests/end_to_end.test.js:1312:20)

src/Client.cc Outdated
@@ -138,7 +138,9 @@ Client::Client(const Napi::CallbackInfo &info) : Napi::ObjectWrap<Client>(info)
if (clientConfig.Has(CFG_AUTH) && clientConfig.Get(CFG_AUTH).IsObject()) {
Napi::Object obj = clientConfig.Get(CFG_AUTH).ToObject();
if (obj.Has(CFG_AUTH_PROP) && obj.Get(CFG_AUTH_PROP).IsObject()) {
Authentication *auth = Authentication::Unwrap(obj.Get(CFG_AUTH_PROP).ToObject());
this->authRef_ = Napi::Persistent(obj.Get(CFG_AUTH_PROP).As<Napi::Object>());
this->authRef_.SuppressDestruct();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SuppressDestruct() suppresses the destruction of authRef_, which is held by the Client instance. Could you try removing this call?

This method is used here:

constructor.SuppressDestruct();

However, constructor is a static variable, whose lifetime is the same with the whole program. But here authRef_'s lifetime is the same with the Client. Suppressing the destruction of authRef_ could cause memory leak.

image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't have time looking deeper for now. But I assume that when the Client destructs, the authRef_ will be destroyed as well unless you call SuppressDestruct().

The original issue happens just because auth is a local variable that is referenced by the client config's authentication field. However, since it's now a field of Client, whose lifetime should be longer than its config. We can now prevent the GC of auth.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @RobertIndie Please help double confirm if there is anything wrong with my analysis above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that's the case then perhaps removing that for the logger object will also stop that other open handle.

I am away this weekend, but will give this a go when I'm back.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your analysis. I think it's better to add a test to verify it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I am on mobile just seen that you have done it and tests passed!

Copy link
Member

@RobertIndie RobertIndie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you help add a test to verify this change?

@shibd shibd added this to the 1.14.0 milestone May 26, 2025
@shibd shibd merged commit 6c01e36 into apache:master Jun 11, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants