Add support for debugging PennyLane (Python) and Catalyst (C++) simultaneously #1712
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -388,3 +388,85 @@ corresponding arguments. | |
| $ /path/to/executable | ||
| MemRef: base@ = 0x64fc9dd5ffc0 rank = 0 offset = 0 sizes = [] strides = [] data = | ||
| 25 | ||
|
|
||
| Mixed-mode debugging of Python and C++ | ||
| ====================================== | ||
|
|
||
| Catalyst supports mixed-mode debugging of Python and/or C++ code when providing the ``debug_compiler=True`` flag to | ||
| the ``@qjit`` decorator. Enabling this option signals to the compiler to wait for an appropriate user-provided signal | ||
| after launching the compiler process. Some notes about use of this support: | ||
|
|
||
| * This functionality requires building Catalyst with debug symbols. This can be achieved via | ||
| ``make all BUILD_TYPE="RelWithDebInfo"``. The debug symbols are only available | ||
| within the Catalyst-owned targets. | ||
| To enable debugging of LLVM and other associated external libraries and binaries, ensure the | ||
| ``BUILD_TYPE_EXT="RelWithDebInfo"`` option is also set when building Catalyst. | ||
| * Launching the C++ debugger requires attaching to a running process. This often requires ``sudo`` privileges on the | ||
| running system. | ||
| * The spawned compiler subprocess immediately issues a ``SIGSTOP`` signal to avoid execution of the compiler. To | ||
| continue execution requires receipt of a ``SIGCONT`` signal after the C++ debugger has attached. | ||
| * To validate if running within an active (Python) debugger session, the function :func:`~.debug.debugger.is_debugger_active` | ||
| can be used. | ||
|
|
||
| The signalling steps can be provided via an active terminal session as | ||
|
|
||
| .. code-block:: shell | ||
|
|
||
| $ kill -s SIGCONT <PID> | ||
|
|
||
| where ``<PID>`` is the process-ID. This can also be issued from an active Python debugger session, such as through VSCode's | ||
| debug terminal as | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| import os, signal | ||
| os.kill(<PID>, signal.SIGCONT) | ||
|
|
||
| To enable support from VSCode, the following configuration files can be used to add debugger configurations for Python, and | ||
| C++. | ||
|
|
||
| .. code-block:: json | ||
| :caption: Filename ``.vscode/launch.json`` | ||
|
|
||
| { | ||
| "version": "0.2.0", | ||
| "configurations": [ | ||
| { | ||
| "name": "(Python): Debug Current Python File", | ||
| "type": "debugpy", | ||
| "request": "launch", | ||
| "program": "${file}", | ||
| "console": "integratedTerminal", | ||
| "justMyCode": false | ||
| }, | ||
| { | ||
| "name": "(C++): Attach To Executing Python Process", | ||
| "type": "cppdbg", | ||
| "request": "attach", | ||
| "program": "${command:python.interpreterPath}", | ||
|
There was a problem hiding this comment. I'm confused by this parameter. I tried removing it but then VSCode complains that "program" is not specified. The thing is though, the process we are attaching to is not running Python, it is running the There was a problem hiding this comment. Does it influence where the source directory is parsed for the debug information? There was a problem hiding this comment.
Hmm, how can I check this? I used the echo path and it still stopped on the breakpoint in my local catalyst installation. |
||
| "processId": "${command:pickProcess}", | ||
| "MIMode": "gdb", | ||
|
There was a problem hiding this comment. Mentioned this in the internal guide, but with the given configuration I get Not sure what I would put for the path though since I can't find There was a problem hiding this comment. Yea, as in the other, I think we need some form of pre-execution script to define a system-dependent argument for this using a custom task There was a problem hiding this comment. For a basic solution, I think it's ok to mention in the instructions to switch to lldb if on mac. |
||
| "setupCommands": [ | ||
| { | ||
| "description": "Enable pretty-printing", | ||
| "text": "-enable-pretty-printing", | ||
| "ignoreFailures": true, | ||
| } | ||
| ] | ||
| }, | ||
| ] | ||
| } | ||
|
|
||
|
|
||
| .. code-block:: json | ||
| :caption: Filename ``.vscode/settings.json`` | ||
|
|
||
| { | ||
| "python.defaultInterpreterPath": "${env:VIRTUAL_ENV}", | ||
| "python.terminal.launchArgs": [], | ||
| } | ||
|
Comment on lines
+463
to
+467
There was a problem hiding this comment. Could you explain these options (and why they are needed)? There was a problem hiding this comment. Sure, will add some references and guidelines on how https://code.visualstudio.com/docs/configure/settings are used here with the debugger. |
||
|
|
||
|
|
||
| Note that on MacOS ``gdb`` will alias ``lldb``, and will continue to function identically | ||
| to ``gdb`` on Linux using the editor's debugging interface. To explicitly use ``lldb`` on Linux, it may be necessary to also | ||
| the `machine-interface driver <https://github.com/lldb-tools/lldb-mi>`_. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,6 +21,7 @@ | |
| import pathlib | ||
| import platform | ||
| import shutil | ||
| import signal | ||
| import subprocess | ||
| import sys | ||
| import tempfile | ||
|
|
@@ -403,7 +404,7 @@ | |
| return cmd | ||
|
|
||
| @debug_logger | ||
| def run_from_ir(self, ir: str, module_name: str, workspace: Directory): | ||
| """Compile a shared object from a textual IR (MLIR or LLVM). | ||
|
|
||
| Args: | ||
|
|
@@ -438,15 +439,36 @@ | |
| output_ir_name = os.path.join(str(workspace), f"{module_name}.ll") | ||
|
|
||
| cmd = self.get_cli_command(tmp_infile_name, output_ir_name, module_name, workspace) | ||
|
|
||
| try: | ||
| if self.options.verbose: | ||
| print(f"[SYSTEM] {' '.join(cmd)}", file=self.options.logfile) | ||
|
|
||
| with subprocess.Popen( | ||
| cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True | ||
| ) as p: | ||
| # Ensure process creation succeeds | ||
| if p.returncode not in {0, None}: | ||
| raise subprocess.CalledProcessError(p.returncode, cmd) | ||
|
|
||
| if self.options.debug_compiler: | ||
| print(f"Compiler PID={p.pid}") | ||
| print( | ||
| f"""Ensure C++ debugger is attached and running before continuing with: | ||
| kill -s SIGCONT {p.pid}""" | ||
| ) | ||
| p.send_signal(signal.SIGSTOP) | ||
|
There was a problem hiding this comment. Do we know the process hasn't done any work yet by the time we get here? There was a problem hiding this comment. It's a good question. The assumption (from most places I've read) is that this should be (reasonably) fine. But if we 100% need to be strict on it, we can modify the process launch options to ensure the signal is the first thing that is hit. There was a problem hiding this comment. Is it just based on hoping that the OS doesn't schedule the subprocess thread until we hit this statement, or is there some other reason it would hold? For instance, does the process only start running once we call There was a problem hiding this comment. So For a quick 'n' dirty example of whether it is fast enough to ensure the process isn't running anything, we can opt for something akin to https://stackoverflow.com/questions/50002804/create-subprocess-in-python-in-suspended-state In practice, use of the import os
import signal
def f():
"pre-execute function to print the new process PID"
print(os.getpid())
print("Starting process")
p = Popen(["/tmp/test"], preexec_fn=f)
print("Running process")
# Stop process after creating the Process and a print call.
# If the `/tmp/test` binary can output before this line is hit,
# we need to stop within the preexec_fn. Otherwise, all good
# to set up the process.
p.send_signal(signal.SIGSTOP)
# Have some wait-time that we control
i = input("enter/return to continue")
# Allow direct input/output from communicate
p.communicate()
# It should be done by now
print("Finished process")
# explicitly kill the process
p.kill()
print("Killed process")Reasoning for favouring not relying on the There was a problem hiding this comment. Thanks for the info! I'll leave it up to you then if we want to use preexec or not, although in the current approach maybe we could at least send the signal before printing? 😅 There was a problem hiding this comment. Fine by me. We can always revert if the above CPython changes make it to a release. Update: As an aside, since the printing will happen from the subprocess, we may have issues with capturing the stdout and stderr for reporting. I'll see if this is something that can be mitigated, but if not, we just use the existing. |
||
|
|
||
| res_stdout, res_stderr = p.communicate() | ||
| # Ensure process execution succeeds | ||
| if p.returncode not in {0, None}: | ||
| raise subprocess.CalledProcessError(p.returncode, cmd, res_stdout, res_stderr) | ||
|
|
||
| if self.options.verbose or os.getenv("ENABLE_DIAGNOSTICS"): | ||
| if res_stdout: | ||
| print(res_stdout.strip(), file=self.options.logfile) | ||
| if res_stderr: | ||
| print(res_stderr.strip(), file=self.options.logfile) | ||
| except subprocess.CalledProcessError as e: # pragma: nocover | ||
| raise CompileError(f"catalyst failed with error code {e.returncode}: {e.stderr}") from e | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # Copyright 2025 Xanadu Quantum Technologies Inc. | ||
|
|
||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
|
|
||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| """ | ||
| This module adds functionality to check if the active Python session | ||
| is being run with an active debugger. | ||
| """ | ||
|
|
||
|
|
||
| import sys | ||
|
|
||
|
|
||
| def is_debugger_active() -> bool: | ||
|
There was a problem hiding this comment. How is this function useful, if we don't use it internally? There was a problem hiding this comment. Looking at this function again, I don't think we should add it to the Catalyst API, it's completely unrelated to Catalyst code. |
||
| """Will return true in active debugger session""" | ||
| return hasattr(sys, "gettrace") and sys.gettrace() | ||
Uh oh!
There was an error while loading. Please reload this page.