heavyrain-lzy · Jun 9, 2026 · Jun 9, 2026 · Jun 9, 2026 · Jun 9, 2026 · Jun 9, 2026
@@ -16,61 +16,27 @@ The platforms and engines in this repository are **reference implementations**
 
 > **Note**: The implementations below are **examples only**. Full production support and maintenance require collaboration with the respective hardware vendors. These serve as templates for vendors to adapt and maintain their own integrations.
 
-| Platform | Device | Communication | Status |
-|----------|--------|---------------|--------|
-| FlagOS | xxx | FlagCX | ✅ Example (requires vendor support) |
-| Intel XPU | Data Center GPU Max / Arc | xccl (oneCCL) | ✅ Example (requires vendor support) |
-| Cambricon MLU | MLU370 / MLU590 | CNCL | ✅ Example (requires vendor support) |
-| MetaX | MetaX GPUs (CUDA-compatible) | NCCL | ✅ Example (requires vendor support) |
-| Huawei NPU | Ascend 910B | HCCL | Built-in (verl core) |
+| Platform | Device | Communication | Status | Doc |
+|----------|--------|---------------|--------|-----|
+| FlagOS | NVIDIA GPU (verified) | FlagCX / NCCL | ✅ Supported | [User Guide](docs/user_guide_flagos/README.md) |
+| Intel XPU | Data Center GPU Max / Arc | xccl (oneCCL) | ✅ Example (requires vendor support) | TBD |
+| Cambricon MLU | MLU370 / MLU590 | CNCL | ✅ Example (requires vendor support) | TBD |
+| MetaX | MetaX GPUs (CUDA-compatible) | NCCL | ✅ Example (requires vendor support) | TBD |
+| Huawei NPU | Ascend 910B | HCCL | Built-in (verl core) | [Ascend Tutorial](https://github.com/verl-project/verl/tree/main/docs/ascend_tutorial) |
 
 
 ## Installation
 
 ```bash
-pip install -e .
+pip install --no-build-isolation -e .
 ```
 
 ## Usage
 
 After `pip install`, the plugin is automatically discovered by verl through the
 `verl.plugins` entry_points group. No additional configuration needed.
 
-### Platform Selection
-
-The platform is auto-detected based on hardware availability. During auto-detection, SMI commands (e.g. `nvidia-smi`, `mx-smi`) are used to distinguish CUDA-compatible hardware. To force a specific platform:
-
-```bash
-export VERL_PLATFORM=intel     # Force Intel XPU
-export VERL_PLATFORM=cambricon # Force Cambricon MLU
-export VERL_PLATFORM=metax     # Force MetaX
-export VERL_PLATFORM=flagos    # Force FlagOS
-```
-
-### FlagOS Configuration
-
-```bash
-# Enable FlagGems operator library
-export USE_FLAGGEMS=true
-
-# Enable FlagCX communication library
-export USE_FLAGCX=1
-
-# Optional: operator whitelist/blacklist
-export TRAINING_FL_FLAGOS_WHITELIST=rmsnorm,layernorm,softmax
-# OR
-export TRAINING_FL_FLAGOS_BLACKLIST=flash_attention
-```
-
-### MetaX Configuration
-
-MetaX GPUs are CUDA-compatible, so they work with `torch.cuda` directly. The plugin uses `mx-smi` to distinguish MetaX hardware from NVIDIA during auto-detection.
-
-```bash
-export VERL_PLATFORM=metax  # or let auto-detection handle it
-
-python -m verl.trainer.main --config your_config.yaml
-```
+For platform-specific usage and configuration, please refer to each platform's documentation in the [Supported Hardware](#supported-hardware-reference-implementations) table above.
 
 ## Architecture
 

@@ -1,46 +1,59 @@
 # FlagOS User Guide
 
-Last updated: 06/04/2026.
+Last updated: 06/09/2026.
 
 ## Introduction
 
-This document describes how to use verl for reinforcement learning training on the FlagOS unified heterogeneous platform.
+This document describes how to use verl for reinforcement learning training with FlagOS multi-chip engine. FlagOS is designed as a unified heterogeneous engine that supports multiple hardware backends through a single interface — it does not register as a separate platform but as an engine vendor (`flagos`) on top of existing device platforms (e.g., `cuda`).
 
 ## Directory Structure
 
 ```
 user_guide_flagos/
 ├── README.md              # This file
 ├── install_guidance.md    # Installation guide
-├── quick_start.md         # Quick start
+├── quick_start.md         # Quick start (NVIDIA example)
+├── env_reference.md       # Environment variables reference
 └── faq.md                 # FAQ and troubleshooting
 ```
 
 ## Getting Started
 
-- [Installation Guide](./install_guidance.md)
-- [Quick Start](./quick_start.md)
-- [FAQ](./faq.md)
+- [Installation Guide](./install_guidance.md) — Docker setup, component installation
+- [Quick Start](./quick_start.md) — Run your first GRPO training job (NVIDIA example)
+- [Environment Variables Reference](./env_reference.md) — Operator dispatch and backend control
+- [FAQ](./faq.md) — Troubleshooting common issues
 
 ## Platform Summary
 
 | Item | Description |
 |------|-------------|
-| Device type | `cuda` (CUDA-compatible) |
+| Type | Engine (not a standalone platform) |
 | Vendor identifier | `flagos` |
-| Communication backend | `flagcx` (or fallback to `nccl`) |
-| Device visibility env var | `CUDA_VISIBLE_DEVICES` |
+| Design | Multi-chip unified engine |
+| Communication backend | `flagcx` (or fallback to device-native, e.g. `nccl`) |
 | Ray resource name | `GPU` |
 | IPC support | Yes |
 
+### Tested Hardware
+
+| Device | Status |
+|--------|--------|
+| NVIDIA GPU | Verified |
+| Ascend NPU | In progress |
+| MetaX | In progress |
+| Iluvatar | In progress |
+
 ## Environment Variables
 
 ```bash
-export VERL_PLATFORM=flagos
+export VERL_ENGINE_VENDOR='flagos'
 export USE_FLAGGEMS=true    # Enable FlagGems operator library
 export USE_FLAGCX=1         # Enable FlagCX communication library
 
 # Optional: operator whitelist/blacklist (choose one)
 export TRAINING_FL_FLAGOS_WHITELIST=rmsnorm,layernorm,softmax
 # export TRAINING_FL_FLAGOS_BLACKLIST=flash_attention
 ```
+
+See [Environment Variables Reference](./env_reference.md) for the full list.
@@ -0,0 +1,99 @@
+# FlagOS Environment Variables Reference
+
+Last updated: 06/09/2026.
+
+> These environment variables apply to all hardware platforms supported by FlagOS (NVIDIA, Ascend, MetaX, etc.) and are not chip-specific.
+
+---
+
+## verl Platform Variables
+
+Environment variables introduced at the verl framework level for platform selection and training behavior:
+
+| Variable | Description | Values | Default |
+|----------|-------------|--------|---------|
+| `VERL_ENGINE_DEVICE` | Device type | `cuda` | - |
+| `VERL_ENGINE_VENDOR` | Vendor identifier | `flagos` | - |
+| `TRAINING_FL_FLAGGEMS_ENABLE` | Enable FlagGems for training | `1` / `0` | `0` |
+| `TRAINING_FL_FLAGOS_WHITELIST` | Training operator whitelist | `rmsnorm,layernorm,softmax` | (none) |
+| `TRAINING_FL_FLAGOS_BLACKLIST` | Training operator blacklist | `flash_attention` | (none) |
+| `USE_FLAGCX` | Enable FlagCX communication library | `1` / `0` | `0` |
+| `RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO` | Ray GPU detection override | `1` / `0` | `1` |
+
+---
+
+## vllm-plugin-FL Inference Dispatch Variables
+
+Controls operator dispatch during inference. For full details, see [dispatch documentation](https://github.com/flagos-ai/vllm-plugin-FL/blob/main/vllm_fl/dispatch/README.md#environment-variables).
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `VLLM_FL_PREFER_ENABLED` | `true` | Global switch; set `false` to disable all dispatch |
+| `VLLM_FL_PREFER` | `flagos` | Preferred backend: `flagos` / `vendor` / `reference` |
+| `VLLM_FL_STRICT` | `0` | Strict mode: `1` = fail on error, `0` = fallback |
+| `VLLM_FL_PER_OP` | (none) | Per-operator backend order, e.g. `rms_norm=vendor\|flagos` |
+| `VLLM_FL_ALLOW_VENDORS` | (none) | Vendor whitelist (comma-separated) |
+| `VLLM_FL_DENY_VENDORS` | (none) | Vendor blacklist (comma-separated) |
+| `USE_FLAGGEMS` | `true` | Enable/disable FlagGems |
+| `VLLM_FL_FLAGOS_WHITELIST` | (none) | FlagGems operator whitelist |
+| `VLLM_FL_FLAGOS_BLACKLIST` | (none) | FlagGems operator blacklist |
+| `VLLM_FL_OOT_ENABLED` | `1` | Enable OOT operator registration |
+| `VLLM_FL_OOT_WHITELIST` | (none) | OOT operator whitelist |
+| `VLLM_FL_OOT_BLACKLIST` | (none) | OOT operator blacklist |
+| `VLLM_FL_CONFIG` | (none) | YAML config file path (complete override) |
+| `VLLM_FL_PLATFORM` | (auto) | Force platform: `cuda` / `ascend` |
+| `VLLM_FL_LOG_LEVEL` | `INFO` | Log level |
+| `VLLM_PLUGINS` | (none) | Specify `fl` when multiple plugins are installed |
+| `FLAGCX_PATH` | (none) | FlagCX installation path; enables flagcx communication backend |
+
+---
+
+## TransformerEngine-FL Training Dispatch Variables
+
+Controls backend selection at the TransformerEngine level. For full details, see [TransformerEngine-FL](https://github.com/flagos-ai/TransformerEngine-FL).
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `TE_FL_PREFER` | `flagos` | Preferred backend: `flagos` / `vendor` / `reference` |
+| `TE_FL_PREFER_VENDOR` | `0` | Prefer vendor backend (legacy) |
+| `TE_FL_STRICT` | `0` | Strict mode; fail without fallback |
+| `TE_FL_ALLOW_VENDORS` | (none) | Vendor whitelist, e.g. `nvidia,amd` |
+| `TE_FL_DENY_VENDORS` | (none) | Vendor blacklist |
+| `TE_FL_PER_OP` | (none) | Per-operator backend order, e.g. `rmsnorm_fwd=vendor:cuda\|default` |
+| `TE_FL_PLUGIN_MODULES` | (none) | Plugin modules to load |
+| `TE_FL_SKIP_CUDA` | `0` | Skip CUDA backend during build |
+| `TEFL_LOG_LEVEL` | `INFO` | Log level |
+
+---
+
+## Common Configuration Examples
+
+Minimal setup (default FlagGems + FlagCX):
+
+```bash
+export VERL_ENGINE_DEVICE='cuda'
+export VERL_ENGINE_VENDOR='flagos'
+export TRAINING_FL_FLAGGEMS_ENABLE=1
+export FLAGCX_PATH=/workspace/FlagCX/
+```
+
+Disable FlagGems and use native CUDA operators:
+
+```bash
+export USE_FLAGGEMS=0
+export TRAINING_FL_FLAGGEMS_ENABLE=0
+```
+
+Disable FlagCX and use NCCL:
+
+```bash
+export USE_FLAGCX=0
+unset FLAGCX_PATH
+```
+
+Debug mode:
+
+```bash
+export VLLM_FL_LOG_LEVEL=DEBUG
+export TEFL_LOG_LEVEL=DEBUG
+```