From 811e48cabf2e80c6c698c130dcfa431db086abf7 Mon Sep 17 00:00:00 2001 From: songguocheng Date: Wed, 12 Aug 2026 20:34:32 +0800 Subject: [PATCH 1/3] Refresh paged GQA4 performance evidence --- README.md | 6 +- docs/flashinfer-parity.md | 10 +- docs/results/README.md | 12 +- ...a4-eager-performance-02faf27-20260812.json | 806 ++++++++++++++++++ docs/roadmap.md | 7 +- website/src/data/site.ts | 8 +- website/src/pages/evidence.astro | 7 +- 7 files changed, 837 insertions(+), 19 deletions(-) create mode 100644 docs/results/h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json diff --git a/README.md b/README.md index 80fbf47a..3c1142f0 100644 --- a/README.md +++ b/README.md @@ -126,13 +126,13 @@ provider-comparison cohort. | Paged decode MHA | B1, KV 1, NHD, D128 | 9.54 µs | 13.77 µs | Oxide 1.44× | | Ragged prefill MHA | Q 16, KV 16, D128 | 8.25 µs | 13.99 µs | Oxide 1.69× | | Ragged prefill GQA4 | Q 32+64, KV 256+1024, D128 | 36.94 µs | 21.93 µs | FlashInfer 1.68× | -| Paged prefill GQA4 | Q 32+64, KV 256+1024, D128 | 57.60 µs | 23.35 µs | FlashInfer 2.47× | +| Paged prefill GQA4 | Q 32+64, KV 256+1024, D128 | 46.60 µs | 23.21 µs | FlashInfer 2.01× | These are CUDA-event measurements of matched operator paths, not isolated kernel, model, or serving results. See the [complete record and raw samples](docs/results/h20-flashinfer-v0.6.17-attention-eager-performance-7f3d08e-20260812.json). -The [optimized paged-GQA4 record](docs/results/h20-flashinfer-v0.6.17-paged-prefill-tiled-gqa4-eager-performance-49290b5-20260812.json) -contains its source-bound two-order samples and before/after comparison. +The [current paged-GQA4 record](docs/results/h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json) +contains separate two-order provider and source-progression cohorts. The [optimized ragged-GQA4 record](docs/results/h20-flashinfer-v0.6.17-ragged-prefill-dual-tile-gqa4-eager-performance-f9b95b0-20260812.json) does the same for the dual-tile path. diff --git a/docs/flashinfer-parity.md b/docs/flashinfer-parity.md index 13ad37c5..5ecf59e1 100644 --- a/docs/flashinfer-parity.md +++ b/docs/flashinfer-parity.md @@ -134,10 +134,12 @@ pinned FlashInfer release. It retains both provider orders and 2,800 raw latency samples. Oxide has lower combined median eager latency in eight stable shapes and FlashInfer in six. -The [optimized paged-GQA4 record](results/h20-flashinfer-v0.6.17-paged-prefill-tiled-gqa4-eager-performance-49290b5-20260812.json) -supersedes only the long paged-GQA4 row. It binds the tiled split-four source, -both provider orders, paged and ragged correctness and Graph gates, and all -four sanitizer tools for both exact runners. +The [current paged-GQA4 record](results/h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json) +supersedes only the long paged-GQA4 row. It keeps provider and source +progression in separate two-order cohorts and binds raw samples, exact runner +and artifact hashes, correctness and Graph gates, and all four sanitizer tools. +The earlier [paged optimization record](results/h20-flashinfer-v0.6.17-paged-prefill-tiled-gqa4-eager-performance-49290b5-20260812.json) +remains its immutable precursor. The [optimized ragged-GQA4 record](results/h20-flashinfer-v0.6.17-ragged-prefill-dual-tile-gqa4-eager-performance-f9b95b0-20260812.json) supersedes only the long ragged-GQA4 row. It binds the dual-tile source and diff --git a/docs/results/README.md b/docs/results/README.md index 4f6eef6b..278e51aa 100644 --- a/docs/results/README.md +++ b/docs/results/README.md @@ -76,6 +76,14 @@ valid-output paged-decode Graph, and all GPUs other than the recorded H20. ## Current-source R2 matched performance +- [Current long paged-GQA4 eager performance against FlashInfer 0.6.17](h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json) + binds source `02faf27`, source `49290b5` as a separate progression cohort, + exact runner and artifact hashes, 400 raw samples, correctness, valid-output + and rejection Graph replay, and all four Compute Sanitizer tools. In the + provider cohort, Oxide measures 46.60 microseconds and FlashInfer 23.21 + microseconds, a 2.01x gap. In the source cohort, current Oxide is 18.45% + lower latency than `49290b5`. + - [Optimized long ragged-GQA4 eager performance against FlashInfer 0.6.17](h20-flashinfer-v0.6.17-ragged-prefill-dual-tile-gqa4-eager-performance-f9b95b0-20260812.json) binds clean source `f9b95b0`, its source-bound parent, exact matched and gate runner hashes, both provider orders, 400 raw latency samples, correctness, @@ -92,12 +100,12 @@ valid-output paged-decode Graph, and all GPUs other than the recorded H20. record retains per-order summaries but not raw samples; it supports the conservative stop decision, not a complete performance qualification. -- [Optimized long paged-GQA4 eager performance against FlashInfer 0.6.17](h20-flashinfer-v0.6.17-paged-prefill-tiled-gqa4-eager-performance-49290b5-20260812.json) +- [First optimized long paged-GQA4 eager performance against FlashInfer 0.6.17](h20-flashinfer-v0.6.17-paged-prefill-tiled-gqa4-eager-performance-49290b5-20260812.json) binds clean source `49290b5`, exact paged and ragged tiled-GQA4 runners, both provider orders, correctness and Graph limits, and all four Compute Sanitizer tools. Oxide moves from 265.66 to 57.60 microseconds on the recorded shape, a 4.61x speedup. FlashInfer remains lower at 23.35 microseconds, a - 2.47x gap. + 2.47x gap. The current paged record above supersedes this exact row. - [Matched BF16 attention eager-provider performance against FlashInfer 0.6.17](h20-flashinfer-v0.6.17-attention-eager-performance-7f3d08e-20260812.json) binds the unchanged product source at merged commit `7f3d08e`, the exact diff --git a/docs/results/h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json b/docs/results/h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json new file mode 100644 index 00000000..8b439e80 --- /dev/null +++ b/docs/results/h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json @@ -0,0 +1,806 @@ +{ + "schema_version": 1, + "date": "2026-08-12", + "claim_level": "matched_eager_provider_performance_and_source_progression", + "status": "passed", + "source": { + "current_commit": "02faf27b116f05831dca7261fb605be13faa2df4", + "current_tree": "c78d8c8051fb1bce4a59f02e0a4fc9e2792b6866", + "published_source_commit": "49290b587838205961b10fc22db108db46018e42", + "published_source_tree": "06ad36ab539dcb44cdbd2b7164ee88607a19a8ed", + "product_source_state": "clean committed source before generated build artifacts", + "compiled_inputs_note": "The current checkout rebuild resolved to the recorded candidate runner hash; commits after f9b95b0 changed evidence or documentation but not compiled Rust inputs.", + "cargo_lock_sha256": "46011524357470cbad8877e81b4ca8dc8731ee82e03cd553803d9cbe9601c01c", + "runner_sha256": { + "matched_current": "d4f5304caed408e15b58ae20f0f6313ddfaffab92ea5f83d82c063f2e33ed568", + "matched_published_source": "b8f28223b7287676d1ded8e49b31eab05e0fa422da9a7ec187db3ff6bd32687e", + "paged_prefill_h20": "143757113f3adf4cd1b0c75c7f9d3fc168c8095dc053828b2546c31f572f2ea0" + }, + "harness_sha256": { + "oxide_matched": "9cbe075d44184ca05c8745affe65b99ec6c0e02c8416f8007adcadef30f06ef0", + "flashinfer_matched": "0d5c6f85840a35239f7c95d1b50010ceeddede98d7bf668625f623a5da2e9e5d", + "summarizer": "363f9c3f4e06097b1e46f5903c55c94f2278f96b80ca3dd2daf36117b2b9629b" + }, + "device_artifact_sha256": { + "ptx": "5a0f603d2a785d4e2aa069cc1f8f7982bb6802d58c20f2ef7767227d6615343c", + "ptxas_sm90a_cubin": "5d1802ddd9c452c5815fea6556510d768aa6b383b73b85972122bfcffc9228e8" + } + }, + "baseline": { + "name": "FlashInfer", + "version": "0.6.17", + "tag": "v0.6.17", + "tag_commit": "a0a6b019b9b27d49d209f85d028a1ae5a9b347d7", + "python_package": "flashinfer-python==0.6.17", + "wheel_source_provenance": "unverified; distribution version only", + "torch": "2.10.0a0+git89fc82e.aml" + }, + "environment": { + "gpu": "NVIDIA H20", + "gpu_uuid": "GPU-3a35e57b-fc54-5620-56ee-deaf5a9c40d3", + "compute_capability": "9.0", + "memory_mib": 97871, + "driver": "535.161.08", + "cuda_toolkit": "13.1.115", + "torch_cuda": "13.1", + "application_clocks_mhz": { + "sm": 1980, + "memory": 2619 + }, + "power_limit_watts": 500, + "pstate": "P0", + "cpu_affinity": [ + 20 + ], + "numa_node": 0, + "omp_num_threads": 1, + "mkl_num_threads": 1, + "flashinfer_nvshmem_library_path": "/usr/lib/x86_64-linux-gnu/nvshmem/13" + }, + "measurement": { + "name": "eager_stream_batch_cuda_event", + "definition": "CUDA event interval around 100 sequential provider calls on one stream, divided by 100", + "warmup_launches": 200, + "launches_per_sample": 100, + "samples_per_order": 50, + "combined_samples_per_subject": 100, + "provider_orders": [ + [ + "oxide-infer", + "flashinfer" + ], + [ + "flashinfer", + "oxide-infer" + ] + ], + "source_orders": [ + [ + "current", + "published_source" + ], + [ + "published_source", + "current" + ] + ], + "stability_gate_percent": 5, + "excluded_from_timed_region": [ + "planning", + "JIT compilation", + "module loading", + "tensor allocation", + "workspace initialization", + "fixture copies", + "final host synchronization" + ], + "included_caveat": "eager host-submission gaps remain inside the CUDA-event interval; not isolated-kernel, Graph, model, or serving timing" + }, + "qualification": { + "paged_tiled_gqa4": { + "status": "passed", + "output_max_abs": 0.00048828125, + "lse_max_abs": 0.000003814697265625, + "commands": 4, + "validator_kernels": 1, + "attention_kernels": 2, + "status_readbacks": 1 + }, + "valid_output_graph": { + "status": "passed", + "commands_per_stage": 4, + "captured_commands": 12, + "replays": 2, + "fixed_bindings": true, + "independent_status_packets": true, + "external_owners_dropped_before_replay": true + }, + "invalid_page_graph": { + "status": "passed", + "replays": 2, + "device_rejected": true, + "graph_reusable": true, + "sentinels_unchanged": true + }, + "workspace_contract": "missing tiled workspace rejected before CUDA submission", + "sanitizers": { + "version": "Compute Sanitizer 2026.2.1.0 build 38334959", + "runner_sha256": "143757113f3adf4cd1b0c75c7f9d3fc168c8095dc053828b2546c31f572f2ea0", + "memcheck": "passed; zero errors and zero leaks", + "racecheck": "passed; zero hazards", + "synccheck": "passed; zero errors", + "initcheck": "passed; zero errors" + } + }, + "resource_usage": { + "compiler": "ptxas CUDA 13.1 targeting sm_90a", + "paged_tiled_gqa4": { + "registers": 168, + "static_shared_bytes": 32768, + "cuobjdump_shared_bytes_including_overhead": 33792, + "stack_bytes": 0, + "spill_store_bytes": 0, + "spill_load_bytes": 0, + "local_bytes": 0, + "launch_bounds_threads": 64 + } + }, + "case": { + "operator": "paged_prefill", + "case": "bf16_paged_prefill_gqa4_b2_q32_64_kv256_1024_qh16_kvh4_d128_p16", + "dtype": "bf16", + "layout": "NHD_D128_page16", + "shape": { + "batch_size": 2, + "head_dim": 128, + "kv_heads": 4, + "max_num_pages": 96, + "nnz_qo": 96, + "page_size": 16, + "query_heads": 16, + "referenced_pages": 80, + "request_kv_lens": [ + 256, + 1024 + ], + "request_qo_lens": [ + 32, + 64 + ] + }, + "fixture_id": "xorshift64_mod2001_bf16_i32_paged_prefill_v1", + "fixture_digests": { + "key_pages": "b4ea1f13ea2fb977", + "last_page_len": "0868f807b519bdad", + "page_indices": "b7671f409220de25", + "page_indptr": "d916e2186be14eb7", + "qo_indptr": "d9b9b2186c6b5e77", + "query": "7f517a5efa9174a5", + "value_pages": "04e7d36418c04659" + }, + "fixture_digest_parity": "identical across both providers, both source revisions, and all recorded orders", + "providers": { + "oxide-infer": { + "version": "1.0.0-alpha.1", + "commit": "02faf27b116f05831dca7261fb605be13faa2df4", + "execution": { + "algorithm": "tiled_gqa4_paged_mma_qk_softmax_pv", + "attention_kernels_per_call": 2, + "causal": "bottom_right", + "commands_per_call": 4, + "metadata_validation": "device_status", + "page_table_location": "device", + "status_readbacks_per_call": 1, + "validator_kernels_per_call": 1 + }, + "correctness": { + "reference": "independent Rust F32 paged-attention formula", + "output_max_abs": 0.00048828125, + "output_max_abs_limit": 0.015625, + "lse_max_abs": 0.000003814697265625, + "lse_max_abs_limit": 0.01, + "output_digest": "48564e4239e7f6e9", + "lse_digest": "62681f9ce2aba81b" + }, + "combined": { + "samples": 100, + "median_us": 46.59984111785889, + "p10_us": 46.227521896362305, + "p90_us": 46.893343925476074, + "min_us": 46.17919921875, + "max_us": 46.96959972381592 + }, + "per_order": { + "oxide_first": { + "samples": 50, + "median_us": 46.287682056427, + "p10_us": 46.20182466506958, + "p90_us": 46.37087821960449, + "min_us": 46.17919921875, + "max_us": 46.45120143890381 + }, + "oxide_second": { + "samples": 50, + "median_us": 46.865761280059814, + "p10_us": 46.80956983566284, + "p90_us": 46.92617654800415, + "min_us": 46.748480796813965, + "max_us": 46.96959972381592 + } + }, + "order_delta_percent": 1.241133345000866, + "raw_samples_us": { + "oxide_first": [ + 46.34175777435303, + 46.37087821960449, + 46.36703968048096, + 46.29439830780029, + 46.2940788269043, + 46.18336200714111, + 46.20192050933838, + 46.227521896362305, + 46.231040954589844, + 46.1846399307251, + 46.27552032470703, + 46.225600242614746, + 46.2662410736084, + 46.27744197845459, + 46.39296054840088, + 46.37087821960449, + 46.293439865112305, + 46.22879981994629, + 46.35712146759033, + 46.35263919830322, + 46.31904125213623, + 46.31711959838867, + 46.25408172607422, + 46.2553596496582, + 46.29663944244385, + 46.45120143890381, + 46.374077796936035, + 46.332478523254395, + 46.322879791259766, + 46.378560066223145, + 46.28928184509277, + 46.35263919830322, + 46.31743907928467, + 46.332478523254395, + 46.235198974609375, + 46.227521896362305, + 46.20351791381836, + 46.20768070220947, + 46.17919921875, + 46.23392105102539, + 46.20096206665039, + 46.19552135467529, + 46.28352165222168, + 46.26719951629639, + 46.31968021392822, + 46.28608226776123, + 46.336002349853516, + 46.2659215927124, + 46.33312225341797, + 46.24320030212402 + ], + "oxide_second": [ + 46.80960178375244, + 46.85344219207764, + 46.94784164428711, + 46.87583923339844, + 46.86592102050781, + 46.87136173248291, + 46.865601539611816, + 46.85376167297363, + 46.876158714294434, + 46.798720359802246, + 46.86783790588379, + 46.88511848449707, + 46.843199729919434, + 46.797118186950684, + 46.86592102050781, + 46.84256076812744, + 46.89311981201172, + 46.87967777252197, + 46.82591915130615, + 46.8720006942749, + 46.90176010131836, + 46.8617582321167, + 46.817917823791504, + 46.74880027770996, + 46.84031963348389, + 46.81407928466797, + 46.83231830596924, + 46.91296100616455, + 46.85760021209717, + 46.748480796813965, + 46.809282302856445, + 46.96959972381592, + 46.96000099182129, + 46.89536094665527, + 46.9379186630249, + 46.93568229675293, + 46.876797676086426, + 46.92512035369873, + 46.895999908447266, + 46.88960075378418, + 46.83680057525635, + 46.85535907745361, + 46.8614387512207, + 46.856322288513184, + 46.824002265930176, + 46.878719329833984, + 46.85760021209717, + 46.87039852142334, + 46.87647819519043, + 46.85376167297363 + ] + } + }, + "flashinfer": { + "version": "0.6.17", + "commit": null, + "execution": { + "algorithm": "flashinfer_batch_paged_prefill_wrapper", + "backend": "fa2", + "causal": "bottom_right", + "disable_split_kv": false, + "kernel_count": { + "status": "unverified" + }, + "page_table_location": "device" + }, + "correctness": { + "lse_bit_mismatches": 1530, + "lse_digest": "5146cf004eee6219", + "lse_domain": "log2", + "lse_max_abs": 0.0003428459167480469, + "lse_max_abs_limit": 0.01, + "lse_reference_digest": "0b9ae4c394a516ac", + "output_bit_mismatches": 93035, + "output_digest": "135e088832c1794b", + "output_max_abs": 0.00048828125, + "output_max_abs_limit": 0.015625, + "output_reference_digest": "6521467467bcd6e5", + "reference": "independent PyTorch F32 paged-attention formula" + }, + "combined": { + "samples": 100, + "median_us": 23.214880228042603, + "p10_us": 22.983328342437744, + "p90_us": 23.364831686019897, + "min_us": 22.936959266662598, + "max_us": 29.826879501342773 + }, + "per_order": { + "flashinfer_first": { + "samples": 50, + "median_us": 22.992639541625977, + "p10_us": 22.97846484184265, + "p90_us": 23.097217082977295, + "min_us": 22.936959266662598, + "max_us": 23.143680095672607 + }, + "flashinfer_second": { + "samples": 50, + "median_us": 23.33392024040222, + "p10_us": 23.30838370323181, + "p90_us": 23.431968212127686, + "min_us": 23.286080360412598, + "max_us": 29.826879501342773 + } + }, + "order_delta_percent": 1.4733694899082086, + "raw_samples_us": { + "flashinfer_first": [ + 23.127999305725098, + 22.98527956008911, + 22.992639541625977, + 22.969279289245605, + 22.987840175628662, + 22.997760772705078, + 22.97760009765625, + 23.004798889160156, + 22.979838848114014, + 22.98527956008911, + 22.992639541625977, + 23.11487913131714, + 22.994558811187744, + 22.97600030899048, + 23.075520992279053, + 22.983040809631348, + 22.99936056137085, + 23.08095932006836, + 23.0019211769104, + 23.136320114135742, + 23.01664113998413, + 22.98464059829712, + 23.083200454711914, + 22.983360290527344, + 22.987840175628662, + 23.051838874816895, + 22.992959022521973, + 22.98975944519043, + 22.957119941711426, + 23.000640869140625, + 22.98719882965088, + 22.936959266662598, + 22.982399463653564, + 22.99168109893799, + 23.001279830932617, + 22.99072027206421, + 22.988159656524658, + 22.97856092453003, + 22.98207998275757, + 23.108160495758057, + 23.004159927368164, + 22.995519638061523, + 23.096001148223877, + 22.989439964294434, + 22.995200157165527, + 23.058879375457764, + 22.98719882965088, + 23.143680095672607, + 23.006720542907715, + 22.99168109893799 + ], + "flashinfer_second": [ + 23.362560272216797, + 23.334720134735107, + 23.337600231170654, + 23.312320709228516, + 23.32927942276001, + 23.33087921142578, + 23.60896110534668, + 23.368000984191895, + 23.320319652557373, + 23.330559730529785, + 23.32479953765869, + 23.324480056762695, + 23.326399326324463, + 23.30847978591919, + 23.326079845428467, + 23.339200019836426, + 23.30751895904541, + 23.33695888519287, + 23.332479000091553, + 23.43168020248413, + 23.361599445343018, + 23.375680446624756, + 23.30336093902588, + 23.448960781097412, + 23.328640460968018, + 23.333120346069336, + 23.35007905960083, + 23.320960998535156, + 23.340160846710205, + 23.337600231170654, + 23.286080360412598, + 23.31615924835205, + 23.3187198638916, + 23.434560298919678, + 23.417279720306396, + 23.344318866729736, + 23.323519229888916, + 23.472959995269775, + 23.327360153198242, + 23.323519229888916, + 23.33791971206665, + 29.826879501342773, + 23.301761150360107, + 23.349759578704834, + 23.349120616912842, + 23.300480842590332, + 23.340799808502197, + 23.364479541778564, + 23.384320735931396, + 23.330559730529785 + ] + } + } + }, + "ranking": { + "status": "stable", + "lower_latency_provider": "flashinfer", + "lower_latency_factor": 2.00732636395721, + "oxide_over_flashinfer_latency_ratio": 2.00732636395721 + } + }, + "source_progression": { + "status": "stable", + "current": { + "commit": "02faf27b116f05831dca7261fb605be13faa2df4", + "combined": { + "samples": 100, + "median_us": 46.95183992385864, + "p10_us": 46.82252740859985, + "p90_us": 47.06719970703125, + "min_us": 46.705918312072754, + "max_us": 47.14591979980469 + }, + "per_order": { + "current_first": { + "samples": 50, + "median_us": 46.8611216545105, + "p10_us": 46.79878377914429, + "p90_us": 46.91779088973999, + "min_us": 46.705918312072754, + "max_us": 46.963839530944824 + }, + "current_second": { + "samples": 50, + "median_us": 47.028799057006836, + "p10_us": 46.965856075286865, + "p90_us": 47.101726055145264, + "min_us": 46.93439960479736, + "max_us": 47.14591979980469 + } + }, + "order_delta_percent": 0.3571787072044447, + "raw_samples_us": { + "current_first": [ + 46.79647922515869, + 46.86272144317627, + 46.81151866912842, + 46.829118728637695, + 46.86783790588379, + 46.80863857269287, + 46.810240745544434, + 46.84031963348389, + 46.84703826904297, + 46.75295829772949, + 46.79903984069824, + 46.70815944671631, + 46.83839797973633, + 46.79296016693115, + 46.84864044189453, + 46.922240257263184, + 46.88064098358154, + 46.89119815826416, + 46.89055919647217, + 46.8236780166626, + 46.84768199920654, + 46.87392234802246, + 46.963839530944824, + 46.87647819519043, + 46.82688236236572, + 46.86272144317627, + 46.91487789154053, + 46.85952186584473, + 46.95712089538574, + 46.889281272888184, + 46.833600997924805, + 46.82079792022705, + 46.88672065734863, + 46.896958351135254, + 46.90688133239746, + 46.864638328552246, + 46.83328151702881, + 46.87583923339844, + 46.91743850708008, + 46.93056106567383, + 46.88896179199219, + 46.85952186584473, + 46.84768199920654, + 46.9209623336792, + 46.89280033111572, + 46.895999908447266, + 46.82271957397461, + 46.84544086456299, + 46.8671989440918, + 46.705918312072754 + ], + "current_second": [ + 46.970558166503906, + 47.14591979980469, + 47.06655979156494, + 47.05984115600586, + 47.100157737731934, + 47.06528186798096, + 47.0252799987793, + 47.06528186798096, + 47.00704097747803, + 47.033281326293945, + 47.03199863433838, + 47.04671859741211, + 47.088961601257324, + 47.12160110473633, + 47.0579195022583, + 47.07871913909912, + 47.076802253723145, + 47.02144145965576, + 47.01183795928955, + 47.01216220855713, + 46.96095943450928, + 47.012481689453125, + 47.03711986541748, + 47.006402015686035, + 46.97279930114746, + 46.95231914520264, + 47.07295894622803, + 46.966400146484375, + 47.02559947967529, + 47.061758041381836, + 47.13183879852295, + 47.115840911865234, + 47.04031944274902, + 47.05535888671875, + 47.128639221191406, + 47.06495761871338, + 47.036800384521484, + 47.0147180557251, + 46.968960762023926, + 46.975998878479004, + 46.95903778076172, + 46.95136070251465, + 47.00064182281494, + 47.01375961303711, + 47.005438804626465, + 46.978559494018555, + 46.93439960479736, + 46.97824001312256, + 47.0147180557251, + 47.0633602142334 + ] + } + }, + "published_source": { + "commit": "49290b587838205961b10fc22db108db46018e42", + "record": "h20-flashinfer-v0.6.17-paged-prefill-tiled-gqa4-eager-performance-49290b5-20260812.json", + "combined": { + "samples": 100, + "median_us": 57.57311820983887, + "p10_us": 57.49974584579468, + "p90_us": 57.65852642059326, + "min_us": 57.46687889099121, + "max_us": 57.67615795135498 + }, + "per_order": { + "published_first": { + "samples": 50, + "median_us": 57.53119945526123, + "p10_us": 57.48284959793091, + "p90_us": 57.56601667404175, + "min_us": 57.46687889099121, + "max_us": 57.60704040527344 + }, + "published_second": { + "samples": 50, + "median_us": 57.620320320129395, + "p10_us": 57.57993698120117, + "p90_us": 57.67052602767944, + "min_us": 57.55936145782471, + "max_us": 57.67615795135498 + } + }, + "order_delta_percent": 0.1547888643449938, + "raw_samples_us": { + "published_first": [ + 57.5651216506958, + 57.49055862426758, + 57.59615898132324, + 57.562241554260254, + 57.550721168518066, + 57.5545597076416, + 57.52543926239014, + 57.56576061248779, + 57.551679611206055, + 57.555198669433594, + 57.514238357543945, + 57.50368118286133, + 57.55807876586914, + 57.56256103515625, + 57.53920078277588, + 57.53376007080078, + 57.50016212463379, + 57.60704040527344, + 57.50783920288086, + 57.49055862426758, + 57.51359939575195, + 57.585601806640625, + 57.562241554260254, + 57.543678283691406, + 57.53215789794922, + 57.53920078277588, + 57.52863883972168, + 57.471680641174316, + 57.53664016723633, + 57.575998306274414, + 57.50847816467285, + 57.550721168518066, + 57.5270414352417, + 57.48288154602051, + 57.484798431396484, + 57.523841857910156, + 57.523841857910156, + 57.511677742004395, + 57.495999336242676, + 57.55328178405762, + 57.568321228027344, + 57.46880054473877, + 57.51967906951904, + 57.478718757629395, + 57.53024101257324, + 57.48256206512451, + 57.55263805389404, + 57.53888130187988, + 57.51391887664795, + 57.46687889099121 + ], + "published_second": [ + 57.58431911468506, + 57.648959159851074, + 57.6470422744751, + 57.66496181488037, + 57.5875186920166, + 57.61695861816406, + 57.6639986038208, + 57.674560546875, + 57.60608196258545, + 57.61343955993652, + 57.624640464782715, + 57.629761695861816, + 57.65823841094971, + 57.642879486083984, + 57.63519763946533, + 57.568960189819336, + 57.67168045043945, + 57.60928153991699, + 57.57023811340332, + 57.601280212402344, + 57.6416015625, + 57.672319412231445, + 57.6416015625, + 57.63167858123779, + 57.595839500427246, + 57.64319896697998, + 57.601280212402344, + 57.60511875152588, + 57.60447978973389, + 57.63296127319336, + 57.599358558654785, + 57.63519763946533, + 57.58016109466553, + 57.661118507385254, + 57.67615795135498, + 57.587199211120605, + 57.56351947784424, + 57.62368202209473, + 57.67615795135498, + 57.58624076843262, + 57.63040065765381, + 57.58687973022461, + 57.67039775848389, + 57.61087894439697, + 57.59039878845215, + 57.66143798828125, + 57.57791996002197, + 57.61439800262451, + 57.55936145782471, + 57.61568069458008 + ] + } + }, + "current_speedup_over_published_source": 1.2262164444078156, + "current_latency_reduction_percent": 18.448329040071208, + "cumulative_changes": [ + "shrink tiled GQA4 query blocks to 8 rows and 64 threads", + "parallelize paged-prefill metadata validation", + "pipeline tiled K and V shared-memory loads" + ] + }, + "accepted_claims": [ + "For the exact matched long paged GQA4 eager contract on the recorded GPU, FlashInfer 0.6.17 has the recorded 2.007x lower combined median latency.", + "The current source is 18.45 percent lower latency than source 49290b5 in the separate source-order cohort.", + "All provider and source order deltas are below five percent.", + "The exact current paged runner passes correctness, valid-output and rejection Graph replay, and all four Compute Sanitizer tools.", + "The current paged tiled kernel has zero ptxas spills and zero local memory." + ], + "excluded_claims": [ + "isolated-kernel timing", + "CUDA Graph performance", + "end-to-end model or serving performance", + "other GPUs or shapes", + "overall library performance", + "FlashInfer wheel source-commit provenance" + ] +} diff --git a/docs/roadmap.md b/docs/roadmap.md index b4444270..36d59958 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -139,9 +139,10 @@ Exit gate: The first current-source matched baseline covers 14 BF16 attention shapes against FlashInfer 0.6.17. All 14 pass the provider-order stability gate; Oxide has lower combined median eager latency in eight and FlashInfer in six. -The first paged long-context GQA4 optimization reduced the recorded Oxide -latency from 265.66 to 57.60 microseconds. FlashInfer remains 2.47x lower on -that exact eager contract. The current ragged optimization reduces its +The current paged long-context GQA4 path measures 46.60 microseconds against +FlashInfer at 23.21 microseconds, a 2.01x gap on that exact eager contract. In +the separate source cohort it is 18.45% lower latency than source `49290b5`. +The current ragged optimization reduces its source-bound parent from 39.42 to 37.04 microseconds. In the separate matched provider cohort, Oxide measures 36.94 microseconds and FlashInfer 21.93 microseconds, a 1.68x gap. diff --git a/website/src/data/site.ts b/website/src/data/site.ts index d5f2fb9d..daf10bb3 100644 --- a/website/src/data/site.ts +++ b/website/src/data/site.ts @@ -1,7 +1,7 @@ export const repositoryUrl = "https://github.com/feichai0017/oxide-infer" export const homeUrl = "https://feichai0017.github.io/oxide-infer/" export const performanceEvidenceUrl = `${repositoryUrl}/blob/main/docs/results/h20-flashinfer-v0.6.17-attention-eager-performance-7f3d08e-20260812.json` -export const pagedPrefillOptimizationEvidenceUrl = `${repositoryUrl}/blob/main/docs/results/h20-flashinfer-v0.6.17-paged-prefill-tiled-gqa4-eager-performance-49290b5-20260812.json` +export const pagedPrefillOptimizationEvidenceUrl = `${repositoryUrl}/blob/main/docs/results/h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json` export const raggedPrefillOptimizationEvidenceUrl = `${repositoryUrl}/blob/main/docs/results/h20-flashinfer-v0.6.17-ragged-prefill-dual-tile-gqa4-eager-performance-f9b95b0-20260812.json` export const navigation = [ @@ -128,9 +128,9 @@ export const performanceRows = [ { name: "Paged prefill · GQA4", shape: "Q 32+64 · KV 256+1024 · D128", - oxideUs: "57.60", - flashinferUs: "23.35", - result: "FlashInfer 2.47× lower", + oxideUs: "46.60", + flashinferUs: "23.21", + result: "FlashInfer 2.01× lower", winner: "flashinfer", }, ] diff --git a/website/src/pages/evidence.astro b/website/src/pages/evidence.astro index 4e2f13e7..0c7d6b0a 100644 --- a/website/src/pages/evidence.astro +++ b/website/src/pages/evidence.astro @@ -127,9 +127,10 @@ import { execution. Tiled GQA4 uses a caller-owned F32 partial workspace.

- The tiled paged-GQA4 path is 4.61× faster than its prior Oxide - baseline for the recorded long shape. FlashInfer remains 2.47× lower - latency, so long MQA and GQA prefill remain optimization targets. + The current paged-GQA4 path is 18.45% lower latency than source + 49290b5 in a separate source cohort. In the matched + provider cohort, Oxide measures 46.60 µs and FlashInfer 23.21 µs, a + 2.01× gap.

Historical token-parallel correctness record From ab858278ee7e1157f9a374d54f21c8eea2fd73f7 Mon Sep 17 00:00:00 2001 From: songguocheng Date: Wed, 12 Aug 2026 20:54:33 +0800 Subject: [PATCH 2/3] fix(website): preserve evidence labels --- website/src/data/site.ts | 2 +- website/src/pages/evidence.astro | 8 ++++---- website/src/pages/index.astro | 4 ++-- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/website/src/data/site.ts b/website/src/data/site.ts index daf10bb3..f18e91fc 100644 --- a/website/src/data/site.ts +++ b/website/src/data/site.ts @@ -1,7 +1,7 @@ export const repositoryUrl = "https://github.com/feichai0017/oxide-infer" export const homeUrl = "https://feichai0017.github.io/oxide-infer/" export const performanceEvidenceUrl = `${repositoryUrl}/blob/main/docs/results/h20-flashinfer-v0.6.17-attention-eager-performance-7f3d08e-20260812.json` -export const pagedPrefillOptimizationEvidenceUrl = `${repositoryUrl}/blob/main/docs/results/h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json` +export const pagedPrefillCurrentEvidenceUrl = `${repositoryUrl}/blob/main/docs/results/h20-flashinfer-v0.6.17-paged-prefill-current-gqa4-eager-performance-02faf27-20260812.json` export const raggedPrefillOptimizationEvidenceUrl = `${repositoryUrl}/blob/main/docs/results/h20-flashinfer-v0.6.17-ragged-prefill-dual-tile-gqa4-eager-performance-f9b95b0-20260812.json` export const navigation = [ diff --git a/website/src/pages/evidence.astro b/website/src/pages/evidence.astro index 0c7d6b0a..8e18fe5b 100644 --- a/website/src/pages/evidence.astro +++ b/website/src/pages/evidence.astro @@ -5,7 +5,7 @@ import { evidenceHighlights, evidenceLevels, performanceEvidenceUrl, - pagedPrefillOptimizationEvidenceUrl, + pagedPrefillCurrentEvidenceUrl, raggedPrefillOptimizationEvidenceUrl, performanceRows, repositoryUrl, @@ -127,7 +127,7 @@ import { execution. Tiled GQA4 uses a caller-owned F32 partial workspace.

- The current paged-GQA4 path is 18.45% lower latency than source + The current paged-GQA4 path is 18.45% lower latency than source{" "} 49290b5 in a separate source cohort. In the matched provider cohort, Oxide measures 46.60 µs and FlashInfer 23.21 µs, a 2.01× gap. @@ -136,8 +136,8 @@ import { Historical token-parallel correctness record · - - Optimized paged-GQA4 performance record + + Current paged-GQA4 performance record diff --git a/website/src/pages/index.astro b/website/src/pages/index.astro index 363806f6..0dde4dad 100644 --- a/website/src/pages/index.astro +++ b/website/src/pages/index.astro @@ -7,7 +7,7 @@ import { milestones, operatorGroups, performanceEvidenceUrl, - pagedPrefillOptimizationEvidenceUrl, + pagedPrefillCurrentEvidenceUrl, raggedPrefillOptimizationEvidenceUrl, performanceRows, performanceSummary, @@ -188,7 +188,7 @@ const base = import.meta.env.BASE_URL.replace(/\/$/, ""); Open all 14 shapes and raw samples ↗ - + Open the optimized paged-GQA4 record ↗ From df395c290ea0836d70cf3826c52970da8f4f1e1f Mon Sep 17 00:00:00 2001 From: songguocheng Date: Wed, 12 Aug 2026 20:55:00 +0800 Subject: [PATCH 3/3] docs(website): align paged evidence link --- website/src/pages/index.astro | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/src/pages/index.astro b/website/src/pages/index.astro index 0dde4dad..ff55e588 100644 --- a/website/src/pages/index.astro +++ b/website/src/pages/index.astro @@ -189,7 +189,7 @@ const base = import.meta.env.BASE_URL.replace(/\/$/, ""); Open all 14 shapes and raw samples ↗ - Open the optimized paged-GQA4 record ↗ + Open the current paged-GQA4 record ↗ Open the optimized ragged-GQA4 record ↗