build(training): standardize on Python 3.12 across manifests, containers, and runtime scripts

[katriendg]

Description

Standardizes the entire Physical AI Toolchain on Python 3.12, raising requires-python to >=3.12 in every pyproject.toml and regenerating all lock files under the new baseline. This unifies a previously fragmented version floor (>=3.11 in some packages, >=3.11,<3.12 in RL) that blocked Dependabot from proposing dependency updates.

Beyond the version bump, the PR eliminates runtime uv pip compile calls from IL training and SIL evaluation scripts, replacing them with installs from pre-compiled requirements.txt files. This matches the existing RL pattern, removes a deployment-time failure mode (Python 3.11 containers resolving 3.12-only wheels), and cuts job startup latency.

Regenerating requirements surfaced pre-existing metadata conflicts in azureml-mlflow (incompatible caps on mlflow-skinny and azure-storage-blob in RL; marshmallow <4.0.0 cap collision in IL). Dropping azureml-mlflow resolved both — the package was a legacy holdover since azure-ai-ml MLClient already provides the MLflow tracking URI directly.

Closes #540

Type of Change

• 🐛 Bug fix (non-breaking change fixing an issue)
• ✨ New feature (non-breaking change adding functionality)
• 💥 Breaking change (fix or feature causing existing functionality to change)
• 📚 Documentation update
• 🏗️ Infrastructure change (Terraform/IaC)
• ♻️ Refactoring (no functional changes)

Component(s) Affected

• infrastructure/terraform/prerequisites/ - Azure subscription setup
• infrastructure/terraform/ - Terraform infrastructure
• infrastructure/setup/ - OSMO control plane / Helm
• workflows/ - Training and evaluation workflows
• training/ - Training pipelines and scripts
• docs/ - Documentation

Changes

Python 3.12 Version Standardization

Raised requires-python to >=3.12 in all six package manifests (root, RL, IL/LeRobot, dataviewer workspace, dataviewer backend, evaluation). Bumped .python-version from 3.11.13 to 3.12.13. Updated the dataviewer backend Dockerfile base image from an erroneous python:3.14-slim to python:3.12-slim, and changed start.sh to create venvs with --python 3.12. Updated copilot-instructions.md references and ruff target-version documentation to py312.

Runtime Compilation Replaced with Pre-compiled Requirements

Replaced dynamic uv pip compile invocations with installs from committed requirements.txt files in three scripts:

• training/il/scripts/submit-azureml-lerobot-training.sh — now installs from training/il/lerobot/requirements.txt with a missing-file guard
• evaluation/sil/infer.sh — same pattern; added PyTorch cu124 fallback index
• evaluation/sil/validate.sh — same pattern; added PyTorch cu124 fallback index

Created training/il/lerobot/requirements.txt (new file) and regenerated training/rl/requirements.txt under Python 3.12 (CUDA packages shifted from cu12 to cu13 variants).

Dependency Cleanup and Version Alignment

Dropped azureml-mlflow from both RL and IL pyprojects. The package caused metadata conflicts (incompatible caps on mlflow-skinny, azure-storage-blob, and marshmallow) and was redundant — azure-ai-ml MLClient resolves the MLflow tracking URI directly. Removed the corresponding azureml.mlflow entry from _REQUIRED_MODULES in launch.py and launch_rsl_rl.py.

Bumped LeRobot from 0.3.3 to 0.4.4 and aligned transitive pins (huggingface-hub, wandb, torchcodec, numpy, packaging) to LeRobot's upstream caps. Bumped IL azure-ai-ml from 1.31.0 to 1.32.0 for parity with RL. IL marshmallow pinned to 3.26.2 (latest 3.x; forced by azure-ai-ml <4.0.0 cap — no known CVEs).

Lock File Regeneration

Regenerated data-management/viewer/backend/uv.lock, data-management/viewer/uv.lock, and root uv.lock under Python 3.12. Systematically removed Python 3.11 (cp311) and PyPy 3.11 (pp311) wheel entries.

Ancillary Fixes

• Replaced deprecated datetime.utcnow() with datetime.now(UTC) in data-management/viewer/backend/tests/storage/conftest.py
• Removed coverage[toml] optional dependency from root pyproject.toml
• Updated docs/contributing/prerequisites.md Python version to 3.12+
• Fixed OSMO installer URL in scripts/security/Test-BinaryFreshness.ps1 to use releases/download path

Related Issues

• Closes build: standardize on Python 3.12 as repository minimum #540
• Related to security(deps): bump the inference-dependencies group across 1 directory with 8 updates #539 (Dependabot blocked by fragmented requires-python)

Testing Performed

• Terraform plan reviewed (no unexpected changes)
• Terraform apply tested in dev environment
• Training scripts tested locally with Isaac Sim
• OSMO workflow submitted successfully
• Smoke tests passed (smoke_test_azure.py)

Documentation Impact

• Documentation updated in this PR

Bug Fix Checklist

N/A — this is a build/dependency change, not a bug fix.

Checklist

• My code follows the project conventions
• Commit messages follow conventional commit format
• I have performed a self-review
• Documentation impact assessed above
• No new linting warnings introduced

Notes

• Breaking change: Python 3.11 support is fully discontinued. All development environments, containers, and CI pipelines require Python 3.12+.
• Only IL marshmallow (3.26.2 vs 4.2.3 in RL) and huggingface-hub/wandb pins move backward relative to the RL package — all forced by azure-ai-ml and LeRobot upstream caps. marshmallow 3.26.2 is the latest 3.x release with no known open CVEs.
• The OSMO installer URL fix in Test-BinaryFreshness.ps1 is unrelated to the Python migration but was included in the same commit.

Follow-up Tasks

• Rebase or re-run Dependabot PR security(deps): bump the inference-dependencies group across 1 directory with 8 updates #539 after this lands — it should resolve cleanly with unified requires-python
• Add CI freshness check that regenerates requirements.txt files and diffs against committed versions to catch pyproject edits that skip regeneration
• Add grep-based regression guard rejecting reintroduction of uv pip compile in IL/evaluation scripts
• Upgrade Isaac Lab container to Python 3.12 (Isaac Sim 5.x) to eliminate the pre-compilation workaround entirely (tracked as Upgrade Isaac Lab container to Isaac Sim 5.x to fix env.close() shutdown hang #114)

[github-actions]

Dependency Review

The following issues were found:

• ✅ 0 vulnerable package(s)
• ✅ 0 package(s) with incompatible licenses
• ✅ 0 package(s) with invalid SPDX license definitions
• ⚠️ 77 package(s) with unknown licenses.
• ⚠️ 6 packages with OpenSSF Scorecard issues.

View full job summary

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.41%. Comparing base (92c5b2e) to head (98be2c4).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #541      +/-   ##
==========================================
+ Coverage   63.68%   66.41%   +2.73%     
==========================================
  Files         248      261      +13     
  Lines       15301    16566    +1265     
  Branches     2106     2272     +166     
==========================================
+ Hits         9744    11003    +1259     
- Misses       5270     5274       +4     
- Partials      287      289       +2     

Flag	Coverage Δ		*Carryforward flag
pester	81.11% <100.00%> (-2.02%)	⬇️	Carriedforward from d18b9ed
pytest-dataviewer	65.12% <ø> (ø)
pytest-dm-tools	?
pytest-evaluation	99.83% <ø> (?)
pytest-fuzz	4.97% <ø> (ø)
pytest-inference	?
pytest-training	81.18% <ø> (ø)
vitest	51.08% <ø> (ø)		Carriedforward from d18b9ed

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines	Coverage Δ
scripts/security/Test-BinaryFreshness.ps1	65.77% <100.00%> (ø)

... and 17 files with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[WilliamBerryiii]

@katriendg @rezatnoMsirhC heads up — pushed two updates to address review feedback and refresh against main:

1. Merged latest main into the branch (commit f5d3446) to clear the stale merge state.
2. Restored the python:3.12-slim SHA256 digest pin (commit 98be2c4) that was inadvertently dropped during the 3.14→3.12 standardization, addressing @rezatnoMsirhC's review comment on data-management/viewer/backend/Dockerfile.

For the digest, I confirmed 3.14 originated from a routine Dependabot bump (PR #481, 3.11→3.14) with no version-specific justification, so 3.12 is preserved per @katriendg's intent. New digest sha256:520153e2deb359602c9cffd84e491e3431d76e7bf95a3255c9ce9433b76ab99a was fetched via docker buildx imagetools inspect python:3.12-slim and resolves to 3.12.13-slim-trixie.

Ready for re-review. 🔒