Upstream sync 2024 06 12

[robertgshaw2-redhat]

Upstream sync 2024 06 12 (#302) - v0.5.0 of vllm

SUMMARY:

• Merge commits from vllm-project@114332b to vllm-project@8f89d72
• Our GCP test instances do not have gcc or clang installed. All of the triton kernels rely on the gcc and clang to generate JITs. These are still disabled (cc @andy-neuma). All are marked with:

@pytest.mark.skip("C compiler not installed in NM automation. "
                  "This codepath follows a triton pathway, which "
                  "JITs using clang or gcc. Since neither are installed "
                  "in our test instances, we need to skip this for now.")

Note that vllm-project@114332b is NOT included in this merge.

COMPARE vs UPSTREAM:

https://github.com/neuralmagic/nm-vllm/compare/upstream-sync-2024-06-11..vllm-project:vllm:v0.5.0

[mgoin]

LGTM!