Implement integration tests in CI pipeline (won't be merged)

[meiyeh123]

Description

Based on the Integration Test requirements, this change adds a new test accuracy on TPU to the CI pipeline. The test covers the Llama-3.1-8B-Instruct and Llama-3.1-70B-Instruct models, modifying the test to support comparing EXPECTED_VALUE. It also allows users to input tensor-parallel-size and model-names parameters for greater flexibility during execution.

Based on the description of the vllm PR at vllm-project/vllm#18800, we have changed the lm_eval version used to git+https://github.com/EleutherAI/lm-evaluation-harness.git@206b7722158f58c35b7ffcd53b035fdbdda5126d#egg=lm-eval[api].

In the future, we'll support collecting expected values on GPUs and comparing them with results from TPUs. We'll achieve this by enabling both GPUs and TPUs to read and write expected JSON files between Buildkite steps. We've already started implementing some of these files.

The testing logic is based on the source code at https://github.com/vllm-project/vllm/blob/839ab00/tests/entrypoints/llm/test_accuracy.py. We are developing based on this source.

Tests

Tested on the Buildkite agent.

Checklist

Before submitting this PR, please make sure:

• I have performed a self-review of my code.
• I have necessary comments in my code, particularly in hard-to-understand areas.

[github-actions]

Description

Start with a short description of what the PR does and how this is a change from
the past.

The rest of the description includes relevant details and context, examples:

• why is this change being made,
• the problem being solved and any relevant context,
• why this is a good solution,
• some information about the specific implementation,
• shortcomings of the solution and possible future improvements.

If the change fixes a bug or a Github issue, please include a link, e.g.,:
FIXES: b/123456
FIXES: #123456

Tests

Please describe how you tested this change, and include any instructions and/or
commands to reproduce.

Checklist

Before submitting this PR, please make sure:

• I have performed a self-review of my code.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have made or will make corresponding changes to any relevant documentation.

[meiyeh123]

this PR is deprecated