Subprocess support not being disabled

[roblourens]

From microsoft/vscode-jupyter#9886 (comment)

I am passing "subProcess": false (and "subprocess": false just to be sure) in with the launch config. Is that the right place? But I still get the debugpyAttach request

Logs
debugpy.zip

[int19h]

Oh, that's because it's an "attach" request! Somehow I missed that earlier. Detouring processes needs to be hooked early on, so it's too late by the time a connection is already live and the request is received. Thus, in this case it is instead configured from the server side via the debugpy module, and it must be done before you call listen() or connect():

debugpy.configure(subProcess=False)

or if using the CLI:

python -m debugpy --configure-subProcess False ...

[PeterM18]

Dumb question, but where would the line debugpy.configure(subProcess=False) go?

[roblourens]

I think that means it would need to be disabled by ipykernel and there is probably nothing I can do. I think for vscode, the solution is just to implement the proper support for debugpyAttach...

[PeterM18]

Thanks roblourens! Evidently the CLI instantiation of debugpy could be called for use inside Code, but this seems jerry-rigged.

[roblourens]

For debugging jupyter via vscode notebooks, I think that ipykernel loads debugpy immediately, and it will be too late for vscode to do anything to configure it.

[int19h]

There's no good workaround for it, but as a hack, you can edit your local copy of debugpy to default to "subProcess": false. To find out where debugpy is loaded from by the Jupyter extension, run this in a notebook cell:

sys.modules['debugpy'].__file__

Once you have the correct path, in debugpy/server/api.py, at the very beginning, there's this bit with all the defaults:

_config = {
    "qt": "none",
    "subProcess": True,
    "python": sys.executable,
}

[PeterM18]

Many, many thanks int19h! I changed "subProcess": True to "subProcess": False in debugpy's api.py, restarted Code, and am now able to run the debugger on multiprocessing code, particularly Pytorch Lightning code.

I imagine I'm not the only person to run into this. Maybe others know the intricacies of Code and debugpy sufficiently to fix this themselves, but I'd have to guess that many wouldn't. Maybe Microsoft should give some warning to users to make the above change so they won't be blocked from debugging increasingly popular multiprocessing code.

[Petros626]

@int19h my looks like this:

# debugpy stable public API consists solely of members of this module that are
# enumerated below.
__all__ = [  # noqa
    "__version__",
    "breakpoint",
    "configure",
    "connect",
    "debug_this_thread",
    "is_client_connected",
    "listen",
    "log_to",
    "trace_this_thread",
    "wait_for_client",
]

import sys

assert sys.version_info >= (3, 7), (
    "Python 3.6 and below is not supported by this version of debugpy; "
    "use debugpy 1.5.1 or earlier."
)


# Actual definitions are in a separate file to work around parsing issues causing
# SyntaxError on Python 2 and preventing the above version check from executing.
from debugpy.public_api import *  # noqa
from debugpy.public_api import __version__

del sys

[int19h]

@Petros626 That's the package entry point, debugpy/__init__.py. The code you need to edit to change the default is in debugpy/server/api.py in the same tree.

[Petros626]

Got it, thank you!

[EnQing626]

@int19h Thank you so much! I was stuck in a situation where I couldn't debug for a whole day and tried a lot of things before I finally found your solution. Especially for just starting to use vscode to debug Pytorch Lightning code remotely, you are a lifesaver.