SIGBUS error with @swc/jest: Intermittent CI failures

[rheaditi]

Hello 👋🏼

We're experiencing an intermittent issue with @swc/jest that's causing our CI builds to fail. We switched from babel-jest to @swc/jest for transpiling our TypeScript project in unit tests, and since then, we've been seeing errors of this format:

  ● Test suite failed to run
    A jest worker process (pid=X) was terminated by another process: signal=SIGBUS, exitCode=null. Operating system logs may contain more information on why this occurred.

A few notable points:-

1. This issue started occurring immediately after switching to @swc/jest, the very first occurrence was on a feature branch introducing this change.
2. It happens in about 1-2.5% of our CI builds (we run upto 3000 builds per day) on our main branch.
3. The error is thrown always on the very first test that is executed, amongst 8 workers (jest is run with maxWorkers=50% on a 16 core machine). The remaining workers seem to be able to run normally, ie is run the remaining tests.
4. Re-running the build usually resolves the issue, making it difficult to reproduce.

I'm primarily looking for help reproducing this issue as without it, it's going to be impossible to debug 😓

We've tried increasing the old space memory for the Node process from 4GB to 8GB and this problem still occurs. We've measured the memory usage of all our tests with jest's --logHeapUsage and --expose-gc but the tests do not exceed usage of 1500MB.

Has anyone else encountered similar issues or have any suggestions for further investigation or potential fixes?
Thank you for your help!

---

Setup / Env details

Dependency Versions

• Node.js – v20.15.1
• @swc/jest – 0.2.37
  • @swc/core – 1.7.35
• jest – 29.7.0

CPU / Memory details of the container running the build

Linux with 64GB, 16 cores

Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Byte Order:                         Little Endian
Address sizes:                      46 bits physical, 57 bits virtual
CPU(s):                             16
On-line CPU(s) list:                0-15
Thread(s) per core:                 1
Core(s) per socket:                 1
Socket(s):                          16
NUMA node(s):                       1
Vendor ID:                          GenuineIntel
CPU family:                         6
Model:                              106
Model name:                         Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz
Stepping:                           6
CPU MHz:                            2899.920
BogoMIPS:                           5799.84
Virtualization:                     VT-x
Hypervisor vendor:                  KVM
Virtualization type:                full
L1d cache:                          512 KiB
L1i cache:                          512 KiB
L2 cache:                           64 MiB
L3 cache:                           256 MiB

swc config

{
  "$schema": "https://swc.rs/schema.json",
  "jsc": {
    "parser": {
      "syntax": "typescript",
      "tsx": true,
      "decorators": true,
      "dynamicImport": true
    },
    "transform": {
      "react": {
        "runtime": "automatic",
        "pragma": "React.createElement",
        "pragmaFrag": "React.Fragment",
        "throwIfNamespace": true,
        "development": false,
        "useBuiltins": false
      }
    },
    "loose": false,
    "externalHelpers": true,
    "experimental": {
      "plugins": [
        [
          "swc_mut_cjs_exports",
          {
            "mutable": true
          }
        ]
      ]
    }
  }
}

[gearsetdave]

We're also experiencing this. I don't have the telemetry to describe it as well as the OP. Will try to gather some, but in the meantime is there anything else we can look at?

[kdy1]

I think the problem is that native node binaries are not a good fit for nodejs worker threads

[kwonoj]

We've tried increasing the old space memory for the Node process from 4GB to 8GB and this problem still occurs. We've measured the memory usage of all our tests with jest's --logHeapUsage and --expose-gc but the tests do not exceed usage of 1500MB.

Worth check maximum memory for the current instance and if jest hits upper bounds of available memory. Actually, increasing heap size can affect this negatively. Say max heap size is 4gb for each process, total maximum memory can be 4 * (N + 1) where N is count of workers and 1 for the host process. If instance is 16 vCPU + 32GB, using 50% (8 vCPU) can hit 4 * 9 = 36, which can exceeds 32.

Not saying every case of sigbus will be this, but I recently hit similar case on 16 vCPU + 32GB only 6 max worker was working reliably, turned out 6 worker can hit 28GB for max, so gives 4GB of buffer to not hitting upper limit and having just 1 more worker can cause the issue.

[kwonoj]

Also this can be very intermittent since factors like jest's transpilation cache can play a lot around memory boundary. So again, not all of sigbus may be memory issue but some cases may related.

[rheaditi]

I see, this seems quite plausible, I will investigate our case further with this in mind!

We didn't go down this path initially since we saw this issue getting hit for the very first test. So we thought something in the setup phase is probably the culprit.

Right now, we are setting each process's memory with (max_memory / workers) * 0.9 but I will try (max_memory / (workers + 1)) * 0.9. The idea with using 90% was to allow for other linux bg processes & certain background tools to also function.

[rheaditi]

This didn't seem to help unfortunately @kwonoj –

(max_memory / (workers + 1)) * 0.9

On 16 cores & 67533 MB memory, setting --max-old-space-size=6753 and still getting the same issue on about 0.6-1% of builds.

jest's transpilation cache can play a lot around memory boundary

Is there something I can try to check if this is the cause?

[kwonoj]

If you can observe the machine's state directly you can confirm easily, otherwise I'm not sure. I recently was able to diagnose this via 3rd party gh runer offer those visibilities.