How to build debug-version jaxlib?

[ronghongbo]

Hello, I would like to build jaxlib with debug info, so that I may trace the execution of Jax into the source CPP files of jaxlib, for better understanding the behavior of the system. I followed the instructions here (https://jax.readthedocs.io/en/latest/developer.html#building-jaxlib-from-source) to build Jaxlib from source, but it does not seem to make Jaxlib debuggable.

For example,

gdb python
r resnet50.py
# After a while, Ctrl-C, and we see this stack:
....
#27 0x00007fff693f7d91 in pybind11::cpp_function::dispatcher(_object*, _object*, _object*) ()
   from /home/u89062/.conda/envs/jax/lib/python3.9/site-packages/jaxlib/mlir/_mlir_libs/_mlir.so
#28 0x00005555556cf969 in cfunction_call (func=0x7fff67de6ae0, args=<optimized out>, kwargs=<optimized out>)
    at /home/sat_bot/base/conda-bld/python_1635297199507/work/Objects/methodobject.c:543
....

Note that there is source file and line number info for Objects/methodobject.c, but there is no such info for jaxlib/mlir/_mlir_libs/_mlir.so

Anyone knows how to accomplish this?

Thanks!
Hongbo

[sogartar]

The documentation states that you should use the script build/build.py. It is a wrapper to a Bazel build that provides the option to pass additional Bazel arguments.
You can enable a debug build by

python build/build.py --bazel_options="--compilation_mode=dbg"

[juuso-oskari]

@ronghongbo Did you manage to get it working? For me simply building with:
python build/build.py --enable_cuda --bazel_options=--override_repository=xla=/xla --bazel_options=--jobs=4 --bazel_options=--compilation_mode=dbg
crashes with:
ERROR: /root/.cache/bazel/_bazel_root/bee4ad1fd43279be7a03b33426e824d5/external/xla/xla/python/BUILD:1257:21: Linking external/xla/xla/python/xla_extension.so failed: (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command (from target @xla//xla/python:xla_extension.so)