Dynamic inference context | UVM only. - #1983

Merged

lmcafee-nvidia merged 85 commits into

NVIDIA:mainfrom

lmcafee-nvidia:lmcafee/dyn-context-uvm-only

Nov 10, 2025

Commits on Sep 9, 2025

copied in unified_memory.py.

ArEsKay3
authored and
lmcafee-nvidia
committed
16609c2
connected arg to context.
lmcafee-nvidia
committed
f32fdf2
minor fixes.
lmcafee-nvidia
committed
beb7f7d
added unit test.
lmcafee-nvidia
committed
c641430
clean up.
lmcafee-nvidia
committed
2d05368
chore: Format files

Mcore Bot
authored and
lmcafee-nvidia
committed
088b320
fix formatting and imports for ci.
lmcafee-nvidia
committed
f322734
chore: Format files

Mcore Bot
authored and
lmcafee-nvidia
committed
330c555
Clean up unified memory code

tdene
authored and
lmcafee-nvidia
committed
4e8148f
Fix pipeline

tdene
authored and
lmcafee-nvidia
committed
55a528e
lint

tdene
authored and
lmcafee-nvidia
committed
ca2cdd3
updated prints/warnings.
lmcafee-nvidia
committed
074e908
Pytest skip if unified memory is not available
tdene
committed
d1e9c4d
chore: Format files
Mcore Bot
committed
8187df3
Address reviewer comments
tdene
committed
18c589e
Merge remote-tracking branch 'lm/lmcafee/dyn-inf-unified-memory' into lmcafee/dyn-inf-unified-memory
tdene
committed
1136dbd

Commits on Sep 16, 2025

Merge remote-tracking branch 'origin/main' into lmcafee/dyn-inf-unified-memory
lmcafee-nvidia
committed
827f055

Commits on Sep 22, 2025

tested different compilation setups.
lmcafee-nvidia
committed
618b3cf
clean up.
lmcafee-nvidia
committed
ea48697
Merge remote-tracking branch 'origin/main' into lmcafee/dyn-inf-unified-memory
lmcafee-nvidia
committed
7561a7f
chore: Format files
Mcore Bot
committed
2eed5a8
Import a mempool-builder instead of global mempool
tdene
committed
b14c607
Fix import issue
tdene
committed
df8ca83
Clean-up, get ready for merge
tdene
committed
ff43bf4
Fix tests
tdene
committed
6922c76

Commits on Sep 30, 2025

sep 30 alpha.
lmcafee-nvidia
committed
c05eb2a

Commits on Oct 1, 2025

oct 1 alpha.
lmcafee-nvidia
committed
eba0a47
fixed max_requests.
lmcafee-nvidia
committed
f604162

Commits on Oct 2, 2025

fixed newly_paused_request_ids.
lmcafee-nvidia
committed
62ce81c
fixed resume_request_count; mostly vectorized.
lmcafee-nvidia
committed
1a74d5c
fixed paused_chunk_counts.
lmcafee-nvidia
committed
9e0b2b3

Commits on Oct 6, 2025

Commits on Oct 7, 2025

passed context to chunk allocator's constructor.
lmcafee-nvidia
committed
f520304
some clean up.
lmcafee-nvidia
committed
12b9866
4gb ran smoothly.
lmcafee-nvidia
committed
185ffbf

Commits on Oct 9, 2025

fixed synthetic prompts to have expected length.
lmcafee-nvidia
committed
f45f3bf

Commits on Oct 13, 2025

fixed resume_request_count.
lmcafee-nvidia
committed
eb7bb9d

Commits on Oct 16, 2025

clean up.
lmcafee-nvidia
committed
7a0934f
minor updates.
lmcafee-nvidia
committed
f93e4de
Merge remote-tracking branch 'origin/main' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
40f08b7
chore: Format files
Mcore Bot
committed
b4d32ef
fixed max_requests rename.
lmcafee-nvidia
committed
55b14a0
Merge remote-tracking branch 'origin/lmcafee/dyn-context-uvm-only' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
2c0d225

Commits on Oct 29, 2025

removed placeholder methods for computing buffer_size and max_tokens.
lmcafee-nvidia
committed
3bdd661

Commits on Oct 30, 2025

re-added unified_memory_level control to dynamic context.
lmcafee-nvidia
committed
559daf3
clean up.
lmcafee-nvidia
committed
be51f16
added uvm level to print.
lmcafee-nvidia
committed
991c95a
format.
lmcafee-nvidia
committed
88e412f
Merge remote-tracking branch 'origin/main' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
5e61868
using correct sampling_params during build_requests().
lmcafee-nvidia
committed
d7c5eae
running again.
lmcafee-nvidia
committed
bf1a342
minor update.
lmcafee-nvidia
committed
73b7041
Merge remote-tracking branch 'origin/main' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
191fe45

Commits on Oct 31, 2025

fixed test_dynamic_context.py.
lmcafee-nvidia
committed
f2548c6
default uvm level == 1; assert(max_tokens >= max_active_requests).
lmcafee-nvidia
committed
fa23ce8
formalized DEFAULT_MAX_TOKENS.
lmcafee-nvidia
committed
c83b373
test simple runs with and without triton kernel.
lmcafee-nvidia
committed
0c387f5
test_dynamic_engine.py works.
lmcafee-nvidia
committed
717dfe9
Merge remote-tracking branch 'main/main' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
152de82
format.
lmcafee-nvidia
committed
e4c6554

Commits on Nov 3, 2025

Commits on Nov 4, 2025

Commits on Nov 7, 2025

Merge remote-tracking branch 'main/main' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
20f1981
minor fix.
lmcafee-nvidia
committed
9b6125d
fixed dynamic context instantiation from static engine.
lmcafee-nvidia
committed
feef5d5
Merge remote-tracking branch 'main/main' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
96c1f00
fixed wandb tests.
lmcafee-nvidia
committed
ee078e5
Merge remote-tracking branch 'main/main' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
5e6f38d

Commits on Nov 10, 2025

Merge remote-tracking branch 'main/main' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
cec714e
integrated mamba updates.
lmcafee-nvidia
committed
7cd2edd
re-added max_tokens arg.
lmcafee-nvidia
committed
4db13d4
fixed test_dynamic_context.py.
lmcafee-nvidia
committed
872acba
second usage of 'with ctx_manager:' fails when using uvm.
lmcafee-nvidia
committed
3ee18fe
using single 'with ctx_manager:' block.
lmcafee-nvidia
committed
e5a3276
Merge remote-tracking branch 'main/main' into lmcafee/dyn-context-uvm-only
lmcafee-nvidia
committed
d493a98
format.
lmcafee-nvidia
committed
f4f60a9