Skip to content

chore(profiling): detect cycles in asyncio #15712

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
KowalskiThomas wants to merge 2 commits into
base: main
Choose a base branch
from
Open

chore(profiling): detect cycles in asyncio #15712

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
30d438e
chore(profiling): detect cycles in asyncio
KowalskiThomas Dec 17, 2025
e7dbf7f
chore(profiling): address review comments
KowalskiThomas Dec 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion ddtrace/internal/datadog/profiling/stack/echion/echion/tasks.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,9 +306,20 @@ TaskInfo::unwind(FrameStack& stack, size_t& upper_python_stack_size)
std::stack<PyObject*> coro_frames;

// Unwind the coro chain
// Detect cycles in the await chain to prevent infinite loops.
// This can happen if the Profiler samples as the Task is running,
// or due to memory corruption/race conditions when reading coroutine pointers.
std::unordered_set<GenInfo*> seen_coros;
for (auto py_coro = this->coro.get(); py_coro != NULL; py_coro = py_coro->await.get()) {
if (py_coro->frame != NULL)
if (seen_coros.find(py_coro) != seen_coros.end()) {
break;
}

seen_coros.insert(py_coro);

if (py_coro->frame != NULL) {
coro_frames.push(py_coro->frame);
}
}

// Total number of frames added to the Stack
Expand Down
9 changes: 8 additions & 1 deletion ddtrace/internal/datadog/profiling/stack/echion/echion/threads.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,11 +300,18 @@ ThreadInfo::unwind_tasks(PyThreadState* tstate)
auto stack_info = std::make_unique<StackInfo>(leaf_task.get().name, leaf_task.get().is_on_cpu);
auto& stack = stack_info->stack;

std::unordered_set<PyObject*> seen_task_origins;
for (auto current_task = leaf_task;;) {
auto& task = current_task.get();

// Check for cycle in task chain
if (seen_task_origins.find(task.origin) != seen_task_origins.end()) {
break;
}
seen_task_origins.insert(task.origin);

// The task_stack_size includes both the coroutines frames and the "upper" Python synchronous frames
size_t task_stack_size = task.unwind(stack, task.is_on_cpu ? upper_python_stack_size : unused);
auto task_stack_size = task.unwind(stack, task.is_on_cpu ? upper_python_stack_size : unused);
if (task.is_on_cpu) {
// Get the "bottom" part of the Python synchronous Stack, that is to say the
// synchronous functions and coroutines called by the Task's outermost coroutine
Expand Down
Loading