Skip to content

Delegate logging_resource_adaptor_impl sync methods to their stream-ordered counterparts #2445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
raja-vardhan wants to merge 1 commit into from
Closed

Delegate logging_resource_adaptor_impl sync methods to their stream-ordered counterparts #2445

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 2 additions & 12 deletions cpp/src/mr/detail/logging_resource_adaptor_impl.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,24 +26,14 @@ logging_resource_adaptor_impl::logging_resource_adaptor_impl(

void* logging_resource_adaptor_impl::allocate_sync(std::size_t bytes, std::size_t alignment)
{
auto const stream = cuda_stream_view{};
try {
auto const ptr = upstream_mr_.allocate(stream, bytes, alignment);
logger_->info("allocate,%p,%zu,%s", ptr, bytes, rmm::detail::format_stream(stream));
return ptr;
} catch (...) {
logger_->info("allocate failure,%p,%zu,%s", nullptr, bytes, rmm::detail::format_stream(stream));
throw;
}
return allocate(cuda_stream_view{}, bytes, alignment);

@wence- wence- Jun 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't change the synchronisation behaviour, which is the thing you're trying to fix.

}
Comment on lines 27 to 30

@coderabbitai coderabbitai Bot Jun 14, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

CRITICAL: Missing stream synchronization in allocate_sync

The synchronous allocation contract requires blocking until the GPU allocation is complete. The current implementation delegates to the stream-ordered allocate but returns immediately without synchronizing, making this method asynchronous rather than synchronous.

Compare to the base class pattern in stream_ordered_memory_resource.hpp (lines 164-170):

auto const stream = cuda_stream_view{};
void* ptr = allocate(stream, bytes, alignment);
stream.synchronize();  // Required!
return ptr;

Why this matters: Callers of allocate_sync expect the allocation to be complete on return. Without synchronization, accessing the memory can race with ongoing GPU operations, causing non-deterministic failures.

🔒 Correct implementation with synchronization 
 void* logging_resource_adaptor_impl::allocate_sync(std::size_t bytes, std::size_t alignment)
 {
-  return allocate(cuda_stream_view{}, bytes, alignment);
+  auto const stream = cuda_stream_view{};
+  void* ptr = allocate(stream, bytes, alignment);
+  stream.synchronize();
+  return ptr;
 }
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/src/mr/detail/logging_resource_adaptor_impl.cpp` around lines 27 - 30,
The `allocate_sync` method in `logging_resource_adaptor_impl::allocate_sync`
delegates to the stream-ordered `allocate` function but fails to synchronize the
stream before returning, making the method asynchronous instead of truly
synchronous as its contract requires. Fix this by capturing the cuda_stream_view
used in the allocate call, and after the allocate call returns, explicitly call
synchronize() on the stream before returning the pointer to ensure the GPU
allocation is complete before the caller gains access to the memory.


void logging_resource_adaptor_impl::deallocate_sync(void* ptr,
std::size_t bytes,
std::size_t alignment) noexcept
{
auto const stream = cuda_stream_view{};
logger_->info("free,%p,%zu,%s", ptr, bytes, rmm::detail::format_stream(stream));
upstream_mr_.deallocate(stream, ptr, bytes, alignment);
deallocate(cuda_stream_view{}, ptr, bytes, alignment);

@wence- wence- Jun 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also wrong: it doesn't synchronise, so it doesn't change the stream synchronisation behaviour at all.

Please follow the pattern in, for example, cuda_memory_resource_impl to see a correct pattern to apply.

}

void* logging_resource_adaptor_impl::allocate(cuda::stream_ref stream,
Expand Down
Loading