NIXLBENCH: update nixlbench/kvbench docs

benchmark/kvbench/README.md

@@ -69,12 +69,11 @@ Options:
   --help                Show this message and exit.
 
 Commands:
-  ct-perftest            Run custom traffic performance test using patterns...
-  io-size                Display IO size information
-  kvcache                Display kvcache information
-  plan                   Display the recommended configuration for nixlbench
-  profile                Run nixlbench
-  sequential-ct-perftest Run sequential custom traffic performance test...
+  ct-perftest             Run custom traffic performance test using...
+  kvcache                 Display kvcache information
+  plan                    Display the recommended configuration for...
+  profile                 Run nixlbench
+  sequential-ct-perftest  Run sequential custom traffic performance test...
 ```
 
 ## Command Line Arguments
@@ -141,8 +140,8 @@ These arguments are used by both `plan` and `profile` commands:
 | `--device_list` | Comma-separated device names (default: all) |
 | `--runtime_type` | Type of runtime to use [ETCD] (default: ETCD) |
 | `--etcd-endpoints` | ETCD server URL for coordination (default: http://localhost:2379) |
-| `--storage_enable_direct` | Enable direct I/O for GDS operations |
-| `--gds_filepath` | File path for GDS operations |
+| `--storage_enable_direct` | Enable direct I/O for storage operations |
+| `--filepath` | File path for storage operations |
 | `--enable_vmm` | Enable VMM memory allocation when DRAM is requested |
 
 ### CTP Command Arguments
@@ -165,40 +164,26 @@ Specific to CTP (Custom Traffic Performance) commands:
 The `plan` command generates and displays recommended `nixlbench` command configurations based on your model architecture and parameters. It helps you prepare optimal benchmark settings without running the benchmark itself.
 
 ```bash
-python main.py plan --model ./examples/model_deepseek_r1.yaml --model_config "./examples/block-tp1-pp8.yaml"
+python main.py plan --model ./examples/model_deepseek_r1.yaml --model_config "./examples/block-tp1-pp8.yaml" --backend POSIX
 ```
 
 #### Profile Command
 
 The `profile` command actually runs the benchmark with the specified configuration using `nixlbench`, collecting performance data across various KV cache operations and access patterns.
 
 ```bash
-python main.py profile --model ./examples/model_deepseek_r1.yaml --model_config "./examples/block-tp1-pp8.yaml"
+python main.py profile --model ./examples/model_deepseek_r1.yaml --model_config "./examples/block-tp1-pp8.yaml" --backend POSIX
 ```
 
 #### KVCache Command
 
 The `kvcache` command analyzes and displays detailed information about the KV cache for a specified model configuration, including model type, sequence lengths, batch sizes, and I/O sizes.
 
 ```bash
-python main.py kvcache --model ./examples/model_deepseek_r1.yaml --model_config "./examples/block-tp1-pp8.yaml"
-Model                  : DEEPSEEK_R1
-Input Sequence Length  : 10000
-Batch Size             : 298
-IO Size                : 1.12 MB
-```
-
-#### IO-Size Command
-
-The `io-size` command displays information about the I/O size requirements for a specified model configuration, helping you understand memory usage and data transfer needs.
-
-```bash
-python main.py io-size --model ./examples/model_deepseek_r1.yaml --model_config "./examples/block-tp1-pp8.yaml"
-Model: DEEPSEEK_R1
-Page Size: 256.0 B
-Input Sequence Length: 10000
-Batch Size: 298
-IO Size: 1.12 MB
+python main.py kvcache --model ./examples/model_deepseek_r1.yaml --model_config "./examples/block-tp1-pp8.yaml" --isl 10000 --page_size 512
+Model          ISL    Num Requests    Batch Size  IO Size      TP    PP    Page Size  Access
+-----------  -----  --------------  ------------  ---------  ----  ----  -----------  --------
+DEEPSEEK_R1  10000              10          1490  2.25 MB       1     8          512  block
 ```
 
 ### CTP Commands

benchmark/kvbench/commands/args.py

@@ -18,9 +18,9 @@
 
 def common_args(func):
     """Decorator for common model arguments"""
-    func = click.option("--model", type=str, help="Model name (e.g., 'llama3.1-8b')")(
-        func
-    )
+    func = click.option(
+        "--model", type=str, help="Path to a model architecture YAML file"
+    )(func)
     func = click.option(
         "--model_config", type=str, help="Path to a single model config YAML file"
     )(func)
@@ -70,7 +70,9 @@ def nixl_bench_args(func):
         help="Destination of the nixl descriptors [file, memory, gpu] (default: memory)",
     )(func)
     func = click.option(
-        "--backend", type=str, help="Communication backend [UCX, UCX_MO] (default: UCX)"
+        "--backend",
+        type=str,
+        help="Communication backend [POSIX, GDS] (default: POSIX)",
     )(func)
     func = click.option(
         "--worker_type",
@@ -156,11 +158,16 @@ def nixl_bench_args(func):
     func = click.option(
         "--storage_enable_direct",
         type=bool,
-        help="Enable direct I/O for storage operations (only used with POSIX backend)",
+        help="Enable direct I/O for storage operations",
         default=False,
     )(func)
     func = click.option(
-        "--gds_filepath", type=str, help="File path for GDS operations"
+        "--filepath", type=str, help="File path for storage operations"
+    )(func)
+    func = click.option(
+        "--posix_api_type",
+        type=str,
+        help="API type for POSIX operations [AIO, URING] (only used with POSIX backend",
     )(func)
     func = click.option(
         "--enable-vmm",

benchmark/kvbench/commands/nixlbench.py

@@ -36,17 +36,21 @@ def __init__(
         enable_pt=False,
         etcd_endpoints="http://localhost:2379",
         storage_enable_direct=False,
-        gds_filepath="",
+        filepath="",
+        gds_batch_pool_size=32,
+        gds_batch_limit=128,
         initiator_seg_type="DRAM",
         enable_vmm=False,
         max_batch_size=None,
         max_block_size=None,
         mode="SG",
+        num_files=1,
         num_initiator_dev=1,
         num_iter=1000,
         num_target_dev=1,
         num_threads=1,
         op_type="WRITE",
+        posix_api_type="AIO",
         runtime_type="ETCD",
         scheme="pairwise",
         start_batch_size=None,
@@ -68,17 +72,21 @@ def __init__(
             enable_pt (bool, optional): Whether to enable peer-to-peer transfer. Defaults to False.
             etcd_endpoints (str, optional): ETCD endpoints for runtime. Defaults to "http://localhost:2379".
             storage_enable_direct (bool, optional): Whether to enable direct I/O for storage operations. Defaults to False.
-            gds_filepath (str, optional): Path for GDS file. Defaults to "".
-            enable_vmm (bool, optional): Whether to use VMM memory allocation. Defaults to False.
+            filepath (str, optional): Path for GDS and POSIX operations. Defaults to "".
+            gds_batch_pool_size (int, optional): Batch pool size for GDS operations. Defaults to 32.
+            gds_batch_limit (int, optional): Batch limit for GDS operations. Defaults to 128.
             initiator_seg_type (str, optional): Type of initiator segment. Defaults to "DRAM".
+            enable_vmm (bool, optional): Whether to use VMM memory allocation. Defaults to False.
             max_batch_size (int, optional): Maximum batch size for testing. Defaults to model_config value.
             max_block_size (int, optional): Maximum block size for testing. Defaults to tp_size * isl.
             mode (str, optional): Benchmarking mode. Defaults to "SG".
+            num_files (int, optional): Number of files. Defaults to 1.
             num_initiator_dev (int, optional): Number of initiator devices. Defaults to 1.
             num_iter (int, optional): Number of iterations. Defaults to 1000.
             num_target_dev (int, optional): Number of target devices. Defaults to 1.
             num_threads (int, optional): Number of threads. Defaults to 1.
             op_type (str, optional): Operation type. Defaults to "WRITE".
+            posix_api_type (str, optional): POSIX API type. Defaults to "AIO".
             runtime_type (str, optional): Runtime type. Defaults to "ETCD".
             scheme (str, optional): Communication scheme. Defaults to "pairwise".
             start_batch_size (int, optional): Starting batch size. Defaults to 1.
@@ -96,17 +104,21 @@ def __init__(
         self.enable_pt = enable_pt
         self.etcd_endpoints = etcd_endpoints
         self.storage_enable_direct = storage_enable_direct
-        self.gds_filepath = gds_filepath
+        self.filepath = filepath
         self.enable_vmm = enable_vmm
+        self.gds_batch_pool_size = gds_batch_pool_size
+        self.gds_batch_limit = gds_batch_limit
         self.initiator_seg_type = initiator_seg_type
         self.max_batch_size = max_batch_size
         self.max_block_size = max_block_size
         self.mode = mode
+        self.num_files = num_files
         self.num_initiator_dev = num_initiator_dev
         self.num_iter = num_iter
         self.num_target_dev = num_target_dev
         self.num_threads = num_threads
         self.op_type = op_type
+        self.posix_api_type = posix_api_type
         self.runtime_type = runtime_type
         self.scheme = scheme
         self.start_batch_size = start_batch_size
@@ -121,25 +133,54 @@ def set_io_size(self, io_size: int):
         self.start_block_size = io_size
         self.max_block_size = io_size
 
+    def _configure_gds(self, source: str, destination: str):
+        if source == "file":
+            # this is a READ from GDS to GPU
+            self.op_type = "READ"
+            self.target_seg_type = "VRAM"
+        elif source == "gpu":
+            # this is a WRITE from GPU to GDS
+            self.op_type = "WRITE"
+            self.target_seg_type = "VRAM"
+        else:
+            raise ValueError(f"Invalid source for GDS: {source}")
+
+    def _configure_posix(self, source: str, destination: str):
+        if source == "file":
+            self.op_type = "READ"
+            self.target_seg_type = "DRAM"
+        elif source == "memory":
+            self.op_type = "WRITE"
+            self.initiator_seg_type = "DRAM"
+        else:
+            raise ValueError(f"Invalid source for POSIX: {source}")
+
     def configure_segment_type(self, backend: str, source: str, destination: str):
-        if backend == "GDS":
-            if source == "file":
-                # this is a READ from GDS to GPU
-                self.op_type = "READ"
-                self.target_seg_type = "VRAM"
-            elif source == "gpu":
-                # this is a WRITE from GPU to GDS
-                self.op_type = "WRITE"
-                self.target_seg_type = "VRAM"
-
-            elif source == "memory":
-                # this is a WRITE from memory to GDS
-                self.op_type = "WRITE"
-                self.initiator_seg_type = "DRAM"
-                self.target_seg_type = "DRAM"
+        if backend.lower() == "gds":
+            self._configure_gds(source, destination)
+        elif backend.lower() == "posix":
+            self._configure_posix(source, destination)
         else:
             raise ValueError(f"Invalid backend: {backend}")
 
+        # if backend == "GDS" or backend == "POSIX":
+        #     if source == "file":
+        #         # this is a READ from GDS to GPU
+        #         self.op_type = "READ"
+        #         self.target_seg_type = "VRAM"
+        #     elif source == "gpu":
+        #         # this is a WRITE from GPU to GDS
+        #         self.op_type = "WRITE"
+        #         self.target_seg_type = "VRAM"
+
+        #     elif source == "memory":
+        #         # this is a WRITE from memory to GDS
+        #         self.op_type = "WRITE"
+        #         self.initiator_seg_type = "DRAM"
+        #         self.target_seg_type = "DRAM"
+        # else:
+        #     raise ValueError(f"Invalid backend: {backend}")
+
     def configure_scheme(self, scheme: str = "pairwise", direction: str = "isl"):
         """
         Configure the scheme based on the model configuration.
@@ -154,6 +195,10 @@ def configure_scheme(self, scheme: str = "pairwise", direction: str = "isl"):
                 self.num_target_dev = 1
 
     def set_batch_size(self, batch_size: int):
+        """
+        Set the batch size for benchmarking.
+        """
+
         self.start_batch_size = batch_size
         self.max_batch_size = batch_size
 
@@ -184,17 +229,21 @@ def _params(self):
             "enable_pt": self.enable_pt,
             "etcd_endpoints": self.etcd_endpoints,
             "storage_enable_direct": self.storage_enable_direct,
-            "gds_filepath": self.gds_filepath,
+            "filepath": self.filepath,
             "enable_vmm": self.enable_vmm,
+            "gds_batch_pool_size": self.gds_batch_pool_size,
+            "gds_batch_limit": self.gds_batch_limit,
             "initiator_seg_type": self.initiator_seg_type,
             "max_batch_size": self.max_batch_size,
             "max_block_size": self.max_block_size,
             "mode": self.mode,
+            "num_files": self.num_files,
             "num_initiator_dev": self.num_initiator_dev,
             "num_iter": self.num_iter,
             "num_target_dev": self.num_target_dev,
             "num_threads": self.num_threads,
             "op_type": self.op_type,
+            "posix_api_type": self.posix_api_type,
             "runtime_type": self.runtime_type,
             "scheme": self.scheme,
             "start_batch_size": self.start_batch_size,
@@ -223,17 +272,21 @@ def defaults():
             "enable_pt": False,
             "etcd_endpoints": "http://localhost:2379",
             "storage_enable_direct": False,
-            "gds_filepath": "",
+            "filepath": "",
             "enable_vmm": False,
+            "gds_batch_pool_size": 32,
+            "gds_batch_limit": 128,
             "initiator_seg_type": "DRAM",
             "max_batch_size": 1,  # ios per gpu
             "max_block_size": 67108864,  # io size
             "mode": "SG",
+            "num_files": 1,
             "num_initiator_dev": 1,
             "num_iter": 1000,
             "num_target_dev": 1,
             "num_threads": 1,
             "op_type": "WRITE",
+            "posix_api_type": "AIO",
             "runtime_type": "ETCD",
             "scheme": "pairwise",
             "start_batch_size": 1,

benchmark/kvbench/docs/tutorial-gds.md

@@ -34,10 +34,6 @@ system:
 
 ## Profiling
 
-> [!NOTE]
-> When using KVBench/NIXLBench, you should perform a WRITE operation before a READ.
-> This is because the files need to be created and populated with data before you can read from them.
-
 ### Step 1: Perform a WRITE operation first (GPU to Storage)
 
 ```bash
@@ -47,15 +43,15 @@ python main.py profile \
   --backend GDS \
   --source gpu \
   --etcd-endpoints "http://localhost:2379" \
-  --gds_filepath /path/to/your/directory \
+  --filepath /path/to/your/directory \
   --num_requests 1
 ```
 
 This command:
 - Uses the DeepSeek model configuration
 - Sets up GDS as the backend
 - Sets the source as GPU memory (`--source gpu`)
-- Specifies the directory for GDS files (`--gds_filepath`)
+- Specifies the directory for GDS files (`--filepath`)
 - Specifies the number of concurrent user requests to simulate (`--num_requests`)
 
 ### Step 2: Perform a READ operation (Storage to GPU)
@@ -69,7 +65,7 @@ python main.py profile \
   --backend GDS \
   --source file \
   --etcd-endpoints "http://localhost:2379" \
-  --gds_filepath /path/to/your/directory \
+  --filepath /path/to/your/directory \
   --num_requests 1
 ```
 
@@ -87,7 +83,7 @@ The profiling results will show:
 ## Troubleshooting
 
 - If you encounter "File not found" errors during READ operations, ensure you've performed a WRITE operation first
-- Check that the directory specified in `--gds_filepath` exists and has proper permissions
+- Check that the directory specified in `--filepath` exists and has proper permissions
 - Verify that your system has GPU Direct Storage support enabled
 - Make sure the ETCD server is running at the specified endpoint