[Feature] Add quant description file for new quant model generated by modelslim

[ganyi1996ppo]

What this PR does / why we need it?

After discussed with MindStudio about the quantization model format, we decide to support another quant format which may used in new modelslim tool, in which case, quantization_config may be removed from the config.json file and quant_model_description.json will be used for quantization configuration.

Does this PR introduce any user-facing change?

How was this patch tested?

[ganyi1996ppo]

@Yikun @wangxiyuan This PR is plan to support the new quant model format generated by the modelslim, please review. BTW, please not merge this PR now, we haven't test the functionality, still needs MindStudio's feedback on this.

[wangxiyuan]

I'm fine with this change. And I think this change is backward compatible. The only problem is to make sure the json file name is named as design.

[Yikun]

Let's wait modelslim feedback and merge, and seems v0.7.3 also need this (via mindie-turbo), right?

[ganyi1996ppo]

ok, I'll cherry-pick those change into v0.7.3 as long as the verification is done by modelslim

[ganyi1996ppo]

Verified on w8a8 model with new config generated by modelslim, the result looks good.

[ganyi1996ppo]

One thing needs to note here, in order to keep compatibility for different version of quantization model generated by modelslim, we should pass the LLM with quantization=ascend by default to enable the quantizaiton feature supported in vllm-ascend, otherwise model load error is expected during vllm engine initialization phase. Below code is a simple example:

import os

from vllm import LLM, SamplingParams

if __name__ == "__main__":
    prompts = [
        "Hello, my name is",
        "The president of the United States is",
        "The capital of France is",
        "The future of AI is",
    ]

    # Create a sampling params object.
    sampling_params = SamplingParams(max_tokens=100, temperature=0.0)
    # Create an LLM.
    llm = LLM(model="qwen2.5_72b_w8a8",
              tensor_parallel_size=4,
              enforce_eager=True,
              trust_remote_code=True,
              max_model_len=1024,
              # needs this key word argument tell engine to run quantization path
              quantization="ascend")

    # Generate texts from the prompts.
    outputs = llm.generate(prompts, sampling_params)
    for output in outputs:
        prompt = output.prompt
        generated_text = output.outputs[0].text
        print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

[Yikun]

LGTM, would you mind also updating the https://vllm-ascend.readthedocs.io/en/latest/tutorials/multi_npu_quantization.html if modelslim has a tag.