Combobox: strange behaviour in testing since Paste v15.3.0

[bsandeepan]

I am facing issues with test cases in Twilio Console repo. Here is the test file:
https://github.com/twilio-internal/console/blob/main/src/applications/Authy/pages/applications/users/index.test.tsx
Line: 77 to 189

I need your help to figure out why Combobox input is behaving differently in my code. Ever since paste update to "^15.3.0", my test cases where I am testing combobox input is behaving strangely and even failing sometimes. Note that the test cases were stable earlier, but now I get flaky behaviour like this: https://buildkite.com/twilio/twilio-internal-console-automated-checks/builds/7814#0183f0a6-a3ed-4023-ac77-81c938fd0106.

Some of these I cannot even reproduce locally. This is what I get locally when I run tests with detectOpenHandles flag:

Command: npm run test -- src/applications/Authy/pages/applications/users/index.test.tsx --detectOpenHandles

console.error
    Warning: Can't perform a React state update on an unmounted component. This is a no-op, but it indicates a memory leak in your application. To fix, cancel all subscriptions and asynchronous tasks in a useEffect cleanup function.
        at Combobox (/Users/twilion/Work/console/node_modules/@twilio-paste/combobox/dist/index.js:1:81738)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at CountryCodeSelect (/Users/twilion/Work/console/src/applications/Authy/shared/components/CountryCodeSelect/index.tsx:14:3)
        at FormProvider (/Users/twilion/Work/console/node_modules/react-hook-form/src/useFormContext.tsx:14:3)
        at CreateUserModalForm (/Users/twilion/Work/console/src/applications/Authy/pages/applications/users/components/CreateUserModal/index.tsx:18:3)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at ModalBody (/Users/twilion/Work/console/node_modules/@twilio-paste/modal/dist/index.js:1:10402)
        at form
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at div
        at /Users/twilion/Work/console/node_modules/@twilio-paste/modal-dialog-primitive/dist/index.js:37:8470
        at Styled(Component) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Animated(Styled(Component)) (/Users/twilion/Work/console/node_modules/@twilio-paste/animation-library/dist/index.js:1:23679)
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at div
        at div
        at /Users/twilion/Work/console/node_modules/@twilio-paste/modal-dialog-primitive/dist/index.js:2:28651
        at div
        at /Users/twilion/Work/console/node_modules/@twilio-paste/modal-dialog-primitive/dist/index.js:2:5332
        at /Users/twilion/Work/console/node_modules/@twilio-paste/modal-dialog-primitive/dist/index.js:37:6338
        at Portal (/Users/twilion/Work/console/node_modules/@twilio-paste/modal-dialog-primitive/dist/index.js:1:21082)
        at /Users/twilion/Work/console/node_modules/@twilio-paste/modal-dialog-primitive/dist/index.js:37:5705
        at Styled(Component) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Animated(Styled(Component)) (/Users/twilion/Work/console/node_modules/@twilio-paste/animation-library/dist/index.js:1:23679)
        at Modal (/Users/twilion/Work/console/node_modules/@twilio-paste/modal/dist/index.js:1:4969)
        at CreateModalAndButton (/Users/twilion/Work/console/src/applications/Authy/shared/components/CreateModalAndButton/index.tsx:28:3)
        at CreateUserButton (/Users/twilion/Work/console/src/applications/Authy/pages/applications/users/components/CreateUserButton/index.tsx:15:19)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at Stack (/Users/twilion/Work/console/node_modules/@twilio-paste/stack/dist/index.js:1:3120)
        at article
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at Card (/Users/twilion/Work/console/node_modules/@twilio-paste/card/dist/index.js:1:2250)
        at EmptyStateCard (/Users/twilion/Work/console/src/applications/Authy/shared/components/EmptyStateCard/index.tsx:11:14)
        at CreateFirstUserMessage
        at ActiveUsersTab (/Users/twilion/Work/console/src/applications/Authy/pages/applications/users/components/ActiveUsersTab/index.tsx:90:44)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at Comp (/Users/twilion/Work/console/node_modules/@twilio-paste/reakit-library/dist/index.js:2:4270)
        at TabPanel (/Users/twilion/Work/console/node_modules/@twilio-paste/tabs/dist/index.js:1:8353)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at TabPanels (/Users/twilion/Work/console/node_modules/@twilio-paste/tabs/dist/index.js:1:8943)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at Tabs (/Users/twilion/Work/console/node_modules/@twilio-paste/tabs/dist/index.js:1:2675)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at UsersTabs (/Users/twilion/Work/console/src/applications/Authy/pages/applications/users/components/UsersTabs/index.tsx:29:39)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at Box (/Users/twilion/Work/console/node_modules/@twilio-paste/box/dist/index.js:1:10413)
        at Flex (/Users/twilion/Work/console/node_modules/@twilio-paste/flex/dist/index.js:1:4088)
        at UsersPage (/Users/twilion/Work/console/src/applications/Authy/pages/applications/users/index.tsx:18:52)
        at div
        at Styled(div) (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39055)
        at ThemeProvider (/Users/twilion/Work/console/node_modules/@twilio-paste/styling-library/dist/index.js:1:39854)
        at ThemeProvider (/Users/twilion/Work/console/node_modules/@twilio-paste/theme/dist/index.js:72:1206)
        at Router (/Users/twilion/Work/console/node_modules/react-router/cjs/react-router.js:99:30)
        at ConnectedRouter (/Users/twilion/Work/console/node_modules/connected-react-router/lib/ConnectedRouter.js:71:7)
        at ConnectedRouterWithContext (/Users/twilion/Work/console/node_modules/connected-react-router/lib/ConnectedRouter.js:195:25)
        at Connect(ConnectedRouterWithContext) (/Users/twilion/Work/console/node_modules/react-redux/lib/components/connectAdvanced.js:233:41)
        at Provider (/Users/twilion/Work/console/node_modules/react-redux/lib/components/Provider.js:21:20)
        at AuthyPageWithContext (/Users/twilion/Work/console/src/applications/Authy/shared/testing-helpers/index.tsx:35:3)

[rmcruzv]

We also have started to experiencing the same bad behavior in all the combobox components. It seems to be related with the help text component

[TheSisb]

This error appears to be because you're checking the Combobox items when the Combobox is closed. In a recent version of Paste, we fixed the component so it doesn't render listbox items until it is opened.

[bsandeepan]

This only resolves one test where we are opening the combobox to select some input. In other two tests (L77 to L148), we are checking the parent component and other input fields where combobox input is only a child. Those two tests are still giving the memory leak warning.

What should we do when we are rendering a parent component with combobox without any intention to interact with the combobox?

[gloriliale]

Hi @bsandeepan! I see, thanks for the update. We're having a hard time figuring out what the problem could be; could you provide us with a reproduction in this codesandbox so we diagnose it better? Also for reference, our tests for combobox are here. Many thanks!

[bsandeepan]

Hi @gloriliale, I have added the combobox here with the tests: https://codesandbox.io/s/combobox-memory-leak-in-tests-forked-ilby2g?file=/index.test.js

I noticed the tests are failing in CSB, but it's related to some library issue. You can run them in your local setup. You need to add detectOpenHandles flag in the command.

[shleewhite]

@bsandeepan Weird about CodeSandbox and tests! I was debugging locally and I think the issue is that you're not wrapping the component in the Paste Theme Provider. Can you try adding that around the Demo component?

[bsandeepan]

@shleewhite That's unlikely. We have a theme wrapper for test cases as well. Maybe we can schedule a meeting in Zoom and review the problem together? I work in IST timezone.

[shleewhite]

Hmm, I'm not sure what is going on with your tests then. You could sign up for our office hours to debug? https://docs.google.com/spreadsheets/d/10vD91zdwr9hjgDOV3HSGnoC0LgvYw-thmn3X_kvW0xk/edit#gid=800758837

[bsandeepan]

The office hours seem super late for my timezone. Do you have any APAC friendly office hours?

[SiTaggart]

Hi There!

Having looked at your tests, and not being able to replicate the issue in an isolated test case, it looks like the issue is somewhere in your component code.

Only one test in that suite causes this issue, and if I comment out a couple of things the error I see is

You are going to have to debug what state change you are performing with CreateUserButton that isn't handled correctly in the test to proceed further. We can't help you with that, unfortunately.

[bsandeepan]

This seems like a different issue to me. I am debugging a memory leak, which points to the recent change in combobox. May you elaborate how your are arriving at this error?

[SiTaggart]

I just isolated one of the tests that opens and closes the modal. Running the test on its own showed the state update warning. Commenting our the close modal action, removed the error.

Seems that when you are removing the modal something is keeping a reference to something in that modal and expecting the state update to be handled, which it isn't because the component has been dismounted.

I have created an isolated test case just using Paste and running a few tests similar to your demo codesandbox. Running the test suite with --detectOpenHandles shows nothing, and with --logHeapUsage shows usual garbage collection. https://github.com/SiTaggart/paste-combobox-test-case

If you can show us that it's combobox related in that test case (fork it if you like) we can investigate further, but right now I can't see that there is a memory leak caused by the component, only tests in the Console test suite.

[bsandeepan]

This is strange. I am not keeping any reference when I close the modal, in fact, it is reused in other pages and they are working without any problem. When I tried to replicate this, I am still getting the error leading to Combobox. Can we schedule a meeting this week around 5-8 PM IST? I think we can come to a better conclusion if we look into the problem together.

[SiTaggart]

If you can provide us an entirely isolated test case that shows the memory leak, we'd be more than happy to debug and fix that in Combobox, but it does need to be isolated from Console. Feel free to fork my repo that I linked above.

There are a lot of tests in the Console test suite that are pouring out these warnings. We can't debug people implementations for them, unfortunately, the team is super busy.

If you're not familiar with async and state updating tests in Jest there are alot of docs on the subject that might be worth reading as it's a very common problem:

• https://reactjs.org/docs/test-utils.html#act
• https://kentcdodds.com/blog/write-fewer-longer-tests#im-still-getting-the-act-warning-even-though-im-using-react-testing-librarys-utilities
• https://github.com/threepointone/react-act-examples/blob/master/sync.md