feat: MLFlow Integration for experiment tracking

[therealnaveenkamal]

What does this PR do ?

Add MLflow integration for experiment tracking and model management in NeMo RL

Issues

Closes #514 - Support for MLflow
This PR addresses the community request for MLflow support alongside existing TensorBoard and Weights & Biases logging options.

Usage

# Install MLflow
uv pip install mlflow

# Run SFT training with MLflow logging
uv run python examples/run_sft.py --config examples/configs/sft_mlflow.yaml

# Start MLflow UI to view results
mlflow ui --host 0.0.0.0 --port 5000

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

This PR adds comprehensive MLflow integration including:

• MLflowLogger class with support for metrics, hyperparameters, and plot logging
• Graceful fallbacks for missing configuration keys
• Example configuration (sft_mlflow.yaml) for SFT training with MLflow
• Updated README with installation and usage instructions
• Fixed test coverage for MLflow plot logging functionality

The integration follows the existing logger architecture and provides a seamless experience for users who want to use MLflow for experiment tracking alongside or instead of TensorBoard and Weights & Biases.

[SahilJain314]

Thanks for contributing! @terrykong to comment on the feature broadly

[terrykong]

thanks @therealnaveenkamal for the contribution! appreciate your help making nemo-rl better!

left some comments below

[terrykong]

a couple other comments

[therealnaveenkamal]

Sure @terrykong , working on your comments. I'll ping you here once I'm ready. Thanks for your support.

[therealnaveenkamal]

@terrykong I think I've addressed all your comments. please review and let me know if further edits are required.

[therealnaveenkamal]

@terrykong / @SahilJain314 not sure why the workflows have failed. do I have to do anything from my side? the lint & the copyright checks passed when I run the workflows locally.

[terrykong]

@therealnaveenkamal I believe the copyright one is orthogonal to your change so you can ignore that (it doesn't gate our CI, but it should CC: @chtruong814)

The linter one fails. can you install the precommit hooks and run it on your branch?

[therealnaveenkamal]

@terrykong thank you. resolved the lint errors.

[terrykong]

thanks @therealnaveenkamal. The CI still says there's some unresolved errors:

[therealnaveenkamal]

@terrykong sorry my bad, I didn't do a lint check after resolving merge conflicts. i think everything should be good now. thanks.

[terrykong]

the linter is failing due to an error that usually indicates the uv.lock needs to be re-locked. i made a PR against your fork to update:

therealnaveenkamal#1

[therealnaveenkamal]

Thanks @terrykong, merged.

[terrykong]

Hi @therealnaveenkamal . I think the CI is having some issues with your PR (CC @chtruong814 ). I'll open a new PR, resolve the issues, and make sure to credit your contribution

[therealnaveenkamal]

Thanks for the support @terrykong !! Looking forward to contributing more.

[terrykong]

Appreciate doing most of the heavy lifting @therealnaveenkamal ! I just pushed it over the finish.

I'll close this since #697 has landed