Skip to content

scripts: download the DAPO dataset the NPU recipe trains on - #2402

Merged
Zhichenzzz merged 1 commit into
mainfrom
shi/npu-dapo-dataset
Aug 12, 2026
Merged

scripts: download the DAPO dataset the NPU recipe trains on#2402
Zhichenzzz merged 1 commit into
mainfrom
shi/npu-dapo-dataset

Conversation

@Shi-Dong

Copy link
Copy Markdown
Contributor

Summary

Follow-up from the dataset-download audit behind #2398/#2399: scripts/run_qwen3_4b_npu.py's prepare() and execute() disagreed about what data and model paths the run uses, so the recipe could not run end to end as written. The script appears to have been copied from the geo3k VLM multi-turn example, and several of its constants were never updated.

  • Dataset: prepare() downloaded VeraIsHere/geo3k_imgurl_processed (used only by a commented-out eval line), while training consumed /root/dataset/dapo-math-17k/dapo-math-17k.jsonl, which nothing downloaded. DATASET_NAME / DATA_ROOT / TRAIN_DATA_PATH now point at zhuzilin/dapo-math-17k (root-level dapo-math-17k.jsonl with prompt + flat label, matching --input-key prompt --label-key label), and --prompt-data references TRAIN_DATA_PATH instead of a hardcoded string, so prepare()'s existence check now guards the file training actually reads.
  • Model path: prepare() downloads to /root/models/{MODEL_NAME}, but --hf-checkpoint and --load pointed at /root/model/Qwen3-4B-Instruct-2507/ — a different directory, and hardcoded so the MILES_SCRIPT_MODEL_NAME override never reached the training args. Both now use f"/root/models/{MODEL_NAME}/".
  • Leftover geo3k naming updated: the wandb group is qwen3-4b-npu instead of geo3k_vlm_multi_turn, and the commented eval stub's dataset label follows suit.

Test plan

  • python -m py_compile scripts/run_qwen3_4b_npu.py (done) and black 24.3.0 clean (done)
  • On an NPU node: prepare() then execute() runs with no missing-file errors, training on DAPO-Math-17k

@Zhichenzzz
Zhichenzzz merged commit dc3bf8f into main Aug 12, 2026
15 of 20 checks passed
@Zhichenzzz
Zhichenzzz deleted the shi/npu-dapo-dataset branch August 12, 2026 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants