Skip to content

Commit e9cbe6e

Browse files
authored
update readme for mxfp mixed precision example (#2302)
Signed-off-by: chensuyue <[email protected]>
1 parent 4ecf8ca commit e9cbe6e

File tree

1 file changed

+12
-3
lines changed
  • examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mix-precision

1 file changed

+12
-3
lines changed

examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mix-precision/README.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Run
22

3-
In this examples, you can verify the accuracy on HPU/CUDA device with emulation of MXFP4, MXFP8, NVFP4 and uNVFP4.
3+
In this example, you can verify the accuracy on HPU/CUDA device with emulation of MXFP4, MXFP8, NVFP4 and uNVFP4.
44

55
## Requirement
66

@@ -9,7 +9,16 @@ In this examples, you can verify the accuracy on HPU/CUDA device with emulation
99
pip install neural-compressor-pt==3.6
1010
# auto-round
1111
pip install auto-round==0.8.0
12-
# others
12+
# other requirements
13+
pip install -r requirements.txt
14+
```
15+
**Before neural-compressor v3.6 and auto-round v0.8.0 release, please install from source for the latest updates:**
16+
```bash
17+
# neural-compressor-pt
18+
INC_PT_ONLY=1 pip install git+https://github.com/intel/[email protected]
19+
# auto-round
20+
pip install git+https://github.com/intel/[email protected]
21+
# other requirements
1322
pip install -r requirements.txt
1423
```
1524

@@ -85,7 +94,7 @@ export no_proxy="localhost, 127.0.0.1, ::1"
8594
curl -X POST http://localhost:7777/v1/completions \
8695
-H "Content-Type: application/json" \
8796
-d '{
88-
"model": "/data0/suyue/Llama-3.3-70B-Instruct-MXFP4",
97+
"model": "/model_path/Llama-3.3-70B-Instruct-MXFP4",
8998
"prompt": "Solve the following math problem step by step: What is 25 + 37? Please answer directly with the result.",
9099
"max_tokens": 100,
91100
"temperature": 0.7,

0 commit comments

Comments
 (0)