Python debugger creates "Dummy-XX" threads causing debugging to fail in callback/waypoint methods

[gnanesh-ks-sndk]

Version: 1.102.2 (user setup)
Commit: c306e94f98122556ca081f527b466015e1bc37b0
Date: 2025-07-22T12:15:48.520Z
Electron: 35.6.0
ElectronBuildId: 11847422
Chromium: 134.0.6998.205
Node.js: 22.15.1
V8: 13.4.114.21-electron.0
OS: Windows_NT x64 10.0.26100

Python Version: 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:34:34) [MSC v.1928 32 bit (Intel)]
Platform: Windows-10-10.0.26100-SP0

Description
When debugging Python code that uses callback methods (particularly firmware waypoint callbacks[ C++ layer call back]), VS Code's Python debugger intermittently creates "Dummy-XX" threads (e.g., "Dummy-18") in the call stack. Once this happens, the debugger becomes unresponsive and cannot continue execution.

1. Set up Python code with callback methods that are triggered asynchronously
2. Set a breakpoint inside the callback method
3. Run the debugger and wait for the callback to be triggered
4. When the breakpoint hits, observe the call stack
5. Attempt to continue execution (F5) or step through code (F10/F11)

Expected Behavior

• Debugger should maintain proper thread context
• "Continue" and "Step" operations should work normally
• Call stack should show actual thread names, not "Dummy-XX"

Actual Behavior

• Call stack shows "Dummy-18" (or similar) instead of real thread name
• Debugger becomes unresponsive to continue/step commands
• Sometimes works once, then fails on subsequent attempts
• Issue is intermittent but reproducible

Debugging Information
When the issue occurs:

• Thread name changes from normal name to "Dummy-XX"
• Thread ID may become None or invalid
• Debugger toolbar buttons become unresponsive
• Debug console shows thread context loss

This issue severely impacts debugging of:

• Firmware validation code with waypoint callbacks

• Event-driven applications with async callbacks

• Any code using threading with callback patterns

• Real-time systems debugging

• Issue is more pronounced with firmware debugging scenarios

• Problem appears to be related to VS Code's thread tracking mechanism

• Other Python IDEs (Wing IDE, PyCharm) handle similar scenarios without issues

• Issue may be related to how VS Code handles callback methods in different thread contexts

• Similar to threading debugging issues reported in other VS Code Python debugging scenarios

• May be related to DAP (Debug Adapter Protocol) thread handling

VS Code's Python debugger should:

1. Maintain proper thread context for callback methods
2. Provide more robust thread tracking for asynchronous operations
3. Implement better recovery mechanisms when thread context is lost
4. Add debugging options for callback/waypoint heavy applications

---

Note: This issue significantly impacts productivity for firmware and embedded systems developers who rely heavily on waypoint/callback debugging patterns.

[rchiodo]

Do you have some sample code that reproduces the problem (on a desktop machine)?

[rchiodo]

You might also try Python 3.12 as I believe that doesn't use dummy threads.

[utkarsh-gupta-sndk]

Do you have some sample code that reproduces the problem (on a desktop machine)?

Tests.zip

Hi @rchiodo,

Thank you for looking into this! I can provide comprehensive reproduction samples and have identified the exact root cause through extensive analysis.

🎯 Root Cause Identified
The issue occurs specifically when native C++ threads invoke Python callbacks (like CVF's boost::python Model.dll), creating threading._DummyThread objects that debugpy cannot properly track during stepping operations.

Key Finding: In our Day to day activity we are using Wing IDE which handles this scenario perfectly, but VS Code's debugpy hangs during step operations in _DummyThread contexts.

📋 Complete Reproduction Suite
I've created multiple reproduction tests of increasing sophistication:

1. Primary Reproduction Test (Most Accurate)

> # TrueNativeCVFThreadingTest.py
> # Uses Windows CreateThread API to create actual foreign threads
> # that call back into Python - exactly like CVF's boost::python
> 
> import threading
> import ctypes
> from ctypes import wintypes
> 
> def critical_waypoint_callback(self, waypoint_type, **kwargs):
>     """*** SET BREAKPOINT HERE ***"""
>     # When this breakpoint hits from native thread,
>     # debugpy stepping commands (F10/F11) hang
>     current_thread = threading.current_thread()
>     print(f"Thread: {current_thread.name}, Type: {type(current_thread)}")
>     # This will show: "Thread: Dummy-1, Type: <class 'threading._DummyThread'>"
>     return {"waypoint": waypoint_type, "processed": True}
> 
> # Creates actual Windows native threads that call back into Python
> CreateThread = ctypes.windll.kernel32.CreateThread
> thread_handle = CreateThread(None, 0, native_callback, None, 0, None)

2. CVF Architecture Simulation🧵 Threading Analysis

# SimplifiedAdvancedCVFTest.py  
# Simulates CVF's ThreadPoolExecutor pattern that creates foreign-like threads

import concurrent.futures

with concurrent.futures.ThreadPoolExecutor(max_workers=2, 
                                          thread_name_prefix="CVF-Foreign") as executor:
    # Submit callback to foreign thread (similar to CVF's boost::python)
    future = executor.submit(self.critical_waypoint_callback, "UM_WRITE")
    # *** SET BREAKPOINT in critical_waypoint_callback() ***
    # Try stepping when breakpoint hits - may hang

Python's Foreign Thread Mechanism:

# From Python's test_threading.py
def test_foreign_thread(self):
    # Creates threading._DummyThread for foreign threads
    self.assertIsInstance(threading._active[tid], threading._DummyThread)

🔍 Real CVF Pattern

# CVF registers Python callbacks with native C++ Model.dll
self.livet = self.testSpace._livet  # Native C++ object
eventKey = self.livet.RegisterFirmwareWaypoint(waypointName, pythonCallback)

# Model.dll creates native Windows threads that invoke pythonCallback
# boost::python bridges C++ threads → Python callbacks
# Result: threading._DummyThread objects that confuse debugpy

📁 Complete Test Files Available
I have a comprehensive reproduction suite ready:

1. TrueNativeCVFThreadingTest.py - Uses Windows CreateThread API (most accurate)
2. SimplifiedAdvancedCVFTest.py - ThreadPoolExecutor simulation (easier to run)
3. CVFBoostPythonSimulationTest.py - Ultimate stress test (6 threads, 60 callbacks)
4. DebugpySteppingHangTest.py - Intensive hang reproduction

VS Code Launch Configurations included for easy testing.

🔧 Reproduction Steps

1. Run any of the test files in VS Code debugger
2. Set breakpoints in critical_waypoint_callback() methods
3. When breakpoint hits from foreign thread context:
   Observe thread name becomes "Dummy-XX"
   Try F10 (Step Over) → Hangs
   Try F11 (Step Into) → Hangs
   Try Shift+F11 (Step Out) → Hangs
4. F5 (Continue) sometimes works, stepping commands hang

💡 Proposed Solution
debugpy needs enhanced threading._DummyThread handling:

1. Detect foreign thread context during callback execution
2. Properly acquire/release GIL and thread state in foreign contexts
3. Maintain execution tracking even when thread wasn't created by Python
4. Graceful stepping recovery in _DummyThread contexts

[utkarsh-gupta-sndk]

You might also try Python 3.12 as I believe that doesn't use dummy threads.

our simulated model + CVF is locked to Python 3.8 due to boost::python compatibility, so it seems not possible to try in 3.12 environments.

[rchiodo]

I can't reproduce the problem using native threads. In fact, breakpoints in the thread callbacks never hit for me. I was wondering how they'd possibly work as they're really native threads, and not python threads, but I suppose that's what the dummy threads are for. And for 3.12 and above the app just hangs in the debugger (well probably because dummy thread support wasn't implemented).

Using the ThreadPoolExecutor, it steps fine:

debugging.mp4

It might be you're using an outdated debugger? Because if I try this with 3.8, the debugger says 3.9 is required. There was a bug a while ago where stepping in threads would not work. That might be what you're hitting (at least in the ThreadPoolExecutor case).

[gramster]

We don't support 3.8; closing