Repository navigation
feat: account for CUDA context memory #304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
iemAnshuman
wants to merge
43
commits into
Project-HAMi:main
from
iemAnshuman:feat-context-accounting
Closed
Changes from 3 commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
df233e6
feat: account for CUDA context memory
iemAnshuman 44e65f0
test: cover the context accounting unit
iemAnshuman 849cf11
fix: defer the charge when the context size is unknown
iemAnshuman 695429f
fix: fall back to the probed context size on every device
iemAnshuman ba096f9
fix: fall back to the probed size only after measuring
iemAnshuman 652d9fc
fix: drop the charge when the primary context is reset
iemAnshuman a34c7d2
fix: keep the probed context size across fork
iemAnshuman 3d2f7cf
fix: lock accounting after the driver call in release
iemAnshuman 2dba259
test: cover rollback mismatch and restore while retained
iemAnshuman cadf987
resolve conflict
iemAnshuman cf4b6ea
test: drive the fork handlers through pthread_atfork
iemAnshuman 037c8e7
chore: remove the ctx_activate flag the retain count replaces
iemAnshuman b8f289a
fix: keep the caller working when the shared region refuses the charge
iemAnshuman 5080bc8
docs: note what the context measurement window can catch
iemAnshuman 967f17c
chore: drop the license headers the repo does not use
iemAnshuman c5d2c13
resolve conflict
iemAnshuman 43198a0
fix: reject an implausible context measurement
iemAnshuman 24ee1b4
fix: preserve context references and use the probe charge
iemAnshuman 2fa5d3f
test: run accounting regressions inside the build container
iemAnshuman 7f99005
test: cover a refused context memory charge
iemAnshuman fdac406
chore: drop the pidfound check in the retain hook
iemAnshuman 0c70a7e
chore: remove the accounted retain variant
iemAnshuman a780672
chore: clear a refused charge directly
iemAnshuman 9c3ab20
chore: remove the unused rollback helper
iemAnshuman 988b025
fix: don't fail a retain we cannot count
iemAnshuman 21d0e51
test: drop nvml fakes that nothing calls
iemAnshuman 76437f0
test: remove two checks that cannot fail
iemAnshuman 746a64a
chore: drop the makefile ctest filter
iemAnshuman 4b3f2d2
chore: fold the fork handlers into context.c
iemAnshuman e09aecc
resolve conflict
iemAnshuman a468ff3
chore: drop an include context.c does not use
iemAnshuman 01caa28
chore: drop the charge local and its stale comment
iemAnshuman e2a41b3
chore: clear fork child accounting with memset
iemAnshuman b21970c
chore: remove unused stdint includes
iemAnshuman b0b880d
test: cover a failed removal on release
iemAnshuman 18920ba
chore: drop the restore helper for failed removals
iemAnshuman d26193d
test: drop an include the fixture does not use
iemAnshuman c7f4c59
test: drop the unused signal.h include
iemAnshuman 2a9242a
test: charge a context retained after its final release
iemAnshuman 53004c6
test: drop a duplicate nested retain check
iemAnshuman bca9b6f
test: reject zero-byte shared region calls in the fixture
iemAnshuman eacad26
test: cover devices outside the accounting range
iemAnshuman bdcfe04
chore: declare the fork handler registration once
iemAnshuman File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| add_library(cuda_mod OBJECT context.c device.c hook.c event.c hook.c memory.c stream.c graph.c) | ||
| add_library(cuda_mod OBJECT context.c context_accounting.c device.c hook.c event.c hook.c memory.c stream.c graph.c) | ||
| target_compile_options(cuda_mod PUBLIC ${LIBRARY_COMPILE_FLAGS}) | ||
| target_link_libraries(cuda_mod PUBLIC nvidia-ml -lcuda) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| /* | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| * | ||
| * Copyright (c) 2026 The HAMi Authors. | ||
| */ | ||
|
|
||
| #include "cuda/context_accounting.h" | ||
|
|
||
| #include <errno.h> | ||
| #include <limits.h> | ||
|
|
||
| static int record_retain(primary_context_accounting_t *state, | ||
| size_t context_bytes, size_t *bytes_to_add, | ||
| int require_charge) { | ||
| if (state == NULL || bytes_to_add == NULL) { | ||
| errno = EINVAL; | ||
| return -1; | ||
| } | ||
| if (state->retain_count == UINT_MAX) { | ||
| errno = EOVERFLOW; | ||
| return -1; | ||
| } | ||
| if (require_charge && state->charged_bytes == 0 && context_bytes == 0) { | ||
| errno = ENODATA; | ||
| return -1; | ||
| } | ||
|
|
||
| *bytes_to_add = 0; | ||
| if (state->charged_bytes == 0 && context_bytes > 0) { | ||
| state->charged_bytes = context_bytes; | ||
| *bytes_to_add = context_bytes; | ||
| } | ||
| state->retain_count++; | ||
| return 0; | ||
| } | ||
|
|
||
| int primary_context_record_retain(primary_context_accounting_t *state, | ||
| size_t context_bytes, | ||
| size_t *bytes_to_add) { | ||
| return record_retain(state, context_bytes, bytes_to_add, 0); | ||
| } | ||
|
|
||
| int primary_context_record_accounted_retain( | ||
| primary_context_accounting_t *state, size_t context_bytes, | ||
| size_t *bytes_to_add) { | ||
| return record_retain(state, context_bytes, bytes_to_add, 1); | ||
| } | ||
|
|
||
| int primary_context_record_release(primary_context_accounting_t *state, | ||
| size_t *bytes_to_remove) { | ||
| if (state == NULL || bytes_to_remove == NULL) { | ||
| errno = EINVAL; | ||
| return -1; | ||
| } | ||
| if (state->retain_count == 0) { | ||
| errno = EINVAL; | ||
| return -1; | ||
| } | ||
|
|
||
| *bytes_to_remove = 0; | ||
| state->retain_count--; | ||
| if (state->retain_count == 0) { | ||
| *bytes_to_remove = state->charged_bytes; | ||
| state->charged_bytes = 0; | ||
| } | ||
| return 0; | ||
| } | ||
|
|
||
| int primary_context_rollback_retain(primary_context_accounting_t *state, | ||
| size_t bytes_to_add) { | ||
| if (state == NULL || state->retain_count == 0) { | ||
| errno = EINVAL; | ||
| return -1; | ||
| } | ||
| if (bytes_to_add > 0 && state->charged_bytes != bytes_to_add) { | ||
| errno = EINVAL; | ||
| return -1; | ||
| } | ||
|
|
||
| state->retain_count--; | ||
| if (bytes_to_add > 0) { | ||
| state->charged_bytes = 0; | ||
| } | ||
| return 0; | ||
| } | ||
|
|
||
| void primary_context_restore_charge(primary_context_accounting_t *state, | ||
| size_t context_bytes) { | ||
| if (state != NULL && state->retain_count == 0 && context_bytes > 0) { | ||
| state->charged_bytes = context_bytes; | ||
| } | ||
| } | ||
|
|
||
| void primary_context_accounting_reset(primary_context_accounting_t *states, | ||
| size_t state_count) { | ||
| size_t i; | ||
|
|
||
| if (states == NULL) { | ||
| return; | ||
| } | ||
| for (i = 0; i < state_count; i++) { | ||
| states[i].retain_count = 0; | ||
| states[i].charged_bytes = 0; | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| /* | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| * | ||
| * Copyright (c) 2026 The HAMi Authors. | ||
| */ | ||
|
|
||
| #ifndef SRC_CUDA_CONTEXT_ACCOUNTING_H_ | ||
| #define SRC_CUDA_CONTEXT_ACCOUNTING_H_ | ||
|
|
||
| #include <stddef.h> | ||
|
|
||
| typedef struct { | ||
| unsigned int retain_count; | ||
| size_t charged_bytes; | ||
| } primary_context_accounting_t; | ||
|
|
||
| /* | ||
| * Record a successful retain or release. A known size is charged once and | ||
| * removed when the final retain is released. | ||
| */ | ||
| int primary_context_record_retain(primary_context_accounting_t *state, | ||
| size_t context_bytes, | ||
| size_t *bytes_to_add); | ||
|
|
||
| /* | ||
| * Record a retain only when its context memory is already charged or a | ||
| * nonzero charge can be applied before the caller reports success. | ||
| */ | ||
| int primary_context_record_accounted_retain( | ||
| primary_context_accounting_t *state, size_t context_bytes, | ||
| size_t *bytes_to_add); | ||
| int primary_context_record_release(primary_context_accounting_t *state, | ||
| size_t *bytes_to_remove); | ||
|
|
||
| /* Undo the most recent retain after its external accounting step fails. */ | ||
| int primary_context_rollback_retain(primary_context_accounting_t *state, | ||
| size_t bytes_to_add); | ||
|
|
||
| /* Restore a charge that could not be removed from shared accounting. */ | ||
| void primary_context_restore_charge(primary_context_accounting_t *state, | ||
| size_t context_bytes); | ||
|
|
||
| /* Clear process-local accounting inherited across fork(). */ | ||
| void primary_context_accounting_reset(primary_context_accounting_t *states, | ||
| size_t state_count); | ||
|
|
||
| #endif // SRC_CUDA_CONTEXT_ACCOUNTING_H_ |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.