feat(model): add bert support.

[bbuschkaemper]

What does this PR do ?

Adds support for the megatron BERT model MegatronBertForMaskedLM.

Changelog

• models/bert/: BertBridge, BertProvider, parameter mapping registry
• conversion/model_bridge.py: fix to tied-embedding filter needed for encoder-only models
• models/__init__.py: register the bert family
• Docs (docs/models/bert/) + runnable example (examples/models/bert/megatron-bert/: conversion, inference, fill-mask)
• Unit tests (bridge, provider) + L0 functional test (toy-model round-trip)

GitHub Actions CI

See the CI section in the Contributing doc for how to trigger the CI. A Nvidia developer will need to approve and trigger the CI for external contributors.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

If you haven't finished some of the above items you can still open "Draft" PR.

Additional Information

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[Copilot]

Pull request overview

Adds first-class support for Megatron-Core’s encoder-only BERT by introducing a new bert model family bridge/provider that converts Hugging Face MegatronBertForMaskedLM checkpoints to/from Megatron-Core BertModel, plus docs, examples, and tests.

Changes:

• Add BertBridge + BertModelProvider and register the new bert family for AutoBridge discovery.
• Fix tied-embedding filtering in conversion to only drop output_layer.weight (preserving standalone output_layer.bias for encoder-only MLM heads).
• Add BERT documentation, runnable examples (conversion + fill-mask), and unit/functional test coverage.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/unit_tests/models/test_model_bridge.py	Adds a regression unit test for tied-embedding filtering to preserve output_layer.bias.
tests/unit_tests/models/bert/test_bert_provider.py	Unit tests for BertModelProvider defaults and provide() wiring.
tests/unit_tests/models/bert/test_bert_bridge.py	Unit tests for HF↔Megatron config mapping, parameter mapping coverage, and export hook behavior.
tests/unit_tests/models/bert/init.py	Establishes BERT unit test package.
tests/functional_tests/test_groups/models/bert/test_bert_conversion.py	Functional tests for toy checkpoint creation, parity, and conversion round-trips (single + multi GPU).
tests/functional_tests/test_groups/models/bert/init.py	Establishes BERT functional test group package.
tests/functional_tests/launch_scripts/h100/active/L0_Launch_models_bert.sh	Adds an L0 launcher for BERT functional tests.
src/megatron/bridge/models/conversion/model_bridge.py	Narrows tied-embedding filtering to drop only output_layer.weight.
src/megatron/bridge/models/bert/bert_provider.py	Implements Megatron-Core BERT provider configuration + instantiation.
src/megatron/bridge/models/bert/bert_bridge.py	Implements HF MegatronBertForMaskedLM ↔ Megatron-Core BertModel bridge and mappings.
src/megatron/bridge/models/bert/init.py	Exposes BERT provider and ensures bridge registration side effects.
src/megatron/bridge/models/init.py	Re-exports/registers the BERT provider at the package level.
README.md	Adds BERT to the supported model families list.
examples/models/bert/megatron-bert/README.md	Documents supported architecture and how to obtain/convert checkpoints.
examples/models/bert/megatron-bert/inference.sh	Adds a fill-mask inference runner for HF checkpoints / round-tripped exports.
examples/models/bert/megatron-bert/fill_mask.py	HF-only fill-mask inference script for sanity checking conversions.
examples/models/bert/megatron-bert/conversion.sh	Example conversion + export + round-trip validation script for BERT.
docs/models/README.md	Adds BERT to the model docs index and encoder-only section.
docs/models/bert/megatron-bert.md	Adds the main BERT variant documentation page.
docs/models/bert/index.md	Adds BERT docs landing page and toctree entry.
docs/index.md	Includes the BERT docs section in the top-level docs index.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[bbuschkaemper]

Important: As part of my effort to add strong support for protein language models to megatron, I also created a few PRs in megatron-core to extend the bert model config options: NVIDIA/Megatron-LM#5540 NVIDIA/Megatron-LM#5687 NVIDIA/Megatron-LM#5690

Once these PRs land and the megatron-core dependency is bumped in megatron-bridge, these config options should be exposed for the BertBridge also, no? Maybe a follow-up PR would be sensible then?

[bbuschkaemper]

@yaoyu-33 If you can find the time, could you take a look at this PR? It is based on #4735 which you also reviewed.

[bbuschkaemper]

Fixed merge conflicts, ready for testing

[bbuschkaemper]

• Rebased branch on current main
• Added additional test coverage for the bert bridge

@yaoyu-33 I saw you adding the needs-more-tests label, are there specific parts not covered rn that I can add?

[bbuschkaemper]

Rebased branch on current main due to new merge conflicts.

[yaoyu-33]

/ok to test d0abe02