Remove deprecated dataset group

[Kipok]

Summary by CodeRabbit

• Chores
  • Removed an unused dataset configuration constant.
  • Relaxed a dependency version constraint to allow newer releases of a packaging component.

[coderabbitai]

📝 Walkthrough

Walkthrough

Removed a public dataset constant and loosened a dependency version pin: DATASET_GROUP was deleted from the AIME26 dataset module; huggingface_hub<1 was changed to huggingface_hub in BFCL_REQUIREMENTS.

Changes

Cohort / File(s)	Summary
Dataset Configuration nemo_skills/dataset/aime26/__init__.py	Removed the public constant DATASET_GROUP = "math". METRICS_TYPE and GENERATION_ARGS left unchanged.
Inference / Eval Dependencies nemo_skills/inference/eval/bfcl.py	Relaxed dependency pin for huggingface_hub in BFCL_REQUIREMENTS (from huggingface_hub<1 to huggingface_hub). No other logic changes.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• Speeding up bfcl requirements installation #1191 — Modifies the same BFCL_REQUIREMENTS constant and huggingface_hub version constraint.

Suggested reviewers

• gwarmstrong

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Remove deprecated dataset group' accurately reflects the main change: removing the DATASET_GROUP constant from nemo_skills/dataset/aime26/init.py.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch igitman/remove-aime-group

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@nemo_skills/inference/eval/bfcl.py`:
- Line 57: The dependency change removed the upper bound on huggingface_hub
which conflicts with the transformers package imported (AutoTokenizer) in
bfcl.py; restore a safe upper bound (e.g., set huggingface_hub<2 in your
dependency listing) to avoid pip resolution conflicts with current transformers
versions, or if you truly need huggingface_hub>=1.0 first upgrade transformers
to a release that supports it before removing the bound; locate the dependency
entry that was changed (the "huggingface_hub" line) and revert/update it
accordingly, referencing the AutoTokenizer import in bfcl.py as the reason for
preserving compatibility.

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

# First, let's look at the file structure and understand the context
cd nemo_skills/inference/eval && head -100 bfcl.py

Repository: NVIDIA-NeMo/Skills

Length of output: 3207

---

🏁 Script executed:

# Check if there are any direct huggingface_hub imports or API usage in this file
rg "from huggingface_hub|import huggingface_hub|huggingface_hub\." nemo_skills/inference/eval/bfcl.py

Repository: NVIDIA-NeMo/Skills

Length of output: 44

---

🏁 Script executed:

# Look at what the BFCL dependencies are
find . -name "*.txt" -o -name "*.toml" -o -name "*.cfg" | xargs grep -l "bfcl\|BFCL" 2>/dev/null

Repository: NVIDIA-NeMo/Skills

Length of output: 85

---

🏁 Script executed:

# Check what transformers currently requires for huggingface_hub
fd -e py . | xargs grep -l "transformers" | head -5

Repository: NVIDIA-NeMo/Skills

Length of output: 280

---

🌐 Web query:

huggingface_hub 1.0 release breaking changes October 2025

💡 Result:

huggingface_hub v1.0.0 was released on October 27, 2025 (with v1.0.1 on Oct 28, 2025). (pypi.org)

Main breaking changes called out for the v1.0 (Oct 2025) release

• Minimum Python version is now 3.9+ (Python 3.8 dropped). (huggingface.co)
• HTTP stack migrated to httpx (dropping requests and aiohttp). Notable impacts include: proxies must be set via HTTP_PROXY/HTTPS_PROXY; configure_http_backend() removed (use set_client_factory() / set_async_client_factory()); HTTP exception ancestry changed (prefer catching HfHubHttpError for compatibility). (huggingface.co)
• CLI breaking change: huggingface-cli command removed in favor of the hf CLI; also the [cli] extra was removed because the CLI ships with the core package. (huggingface.co)
• Git-based workflow removed: the legacy Repository class is gone; use HTTP-based methods like upload_file() / create_commit() instead. (huggingface.co)
• Auth/token handling change: legacy HfFolder token management replaced by explicit login(), logout(), get_token(). (huggingface.co)
• Inference API change: InferenceApi replaced by InferenceClient. (huggingface.co)
• Transfer backend change: hf_transfer replaced by hf_xet (file transfer stack update). (huggingface.co)
• TensorFlow + Keras 2.x support removed: TF-related extras/utilities removed; Keras 2.x integration utilities removed (recommend Keras 3.x). (huggingface.co)

Citations:

• 1: https://pypi.org/project/huggingface-hub/?utm_source=openai
• 2: https://huggingface.co/docs/huggingface_hub/main/concepts/migration?utm_source=openai
• 3: https://huggingface.co/docs/huggingface_hub/main/concepts/migration?utm_source=openai
• 4: https://huggingface.co/blog/huggingface-hub-v1?utm_source=openai
• 5: https://huggingface.co/blog/huggingface-hub-v1?utm_source=openai
• 6: https://huggingface.co/blog/huggingface-hub-v1?utm_source=openai
• 7: https://huggingface.co/blog/huggingface-hub-v1?utm_source=openai
• 8: https://huggingface.co/blog/huggingface-hub-v1?utm_source=openai
• 9: https://huggingface.co/docs/huggingface_hub/main/concepts/migration?utm_source=openai

---

🌐 Web query:

huggingface_hub 1.0 requests httpx migration breaking changes

💡 Result:

In huggingface_hub v1.0, the library migrated from requests (sync) + aiohttp (async) to httpx (sync + async), and dropped the requests and aiohttp dependencies. This introduces a few user-visible breaking changes to be aware of: [1]

• Proxies: “per-method” proxy parameters are no longer supported; proxies must be set globally via HTTP_PROXY / HTTPS_PROXY environment variables. [1]
• Custom HTTP backend configuration: configure_http_backend() was removed; use set_client_factory() / set_async_client_factory() instead. [1]
• Exceptions / error handling:
  • Hub HTTP errors no longer inherit from requests.HTTPError; they now inherit from httpx.HTTPError. Prefer catching huggingface_hub.HfHubHttpError for compatibility across v0.x and v1.x. [1]
  • SSLError is no longer a distinct type (in httpx it becomes a generic connection error, e.g. httpx.ConnectError). [1]
  • LocalEntryNotFoundError no longer inherits from HTTPError; there’s a new EntryNotFoundError split into local vs remote, and only the remote one inherits from HTTPError. [1]
• Inference clients:
  • InferenceClient can be used as a context manager (useful for streaming to ensure connections close cleanly). [1]
  • AsyncInferenceClient(..., trust_env=...) → trust_env was removed (environment variables are trusted by default in httpx); if you need different behavior, configure it via the client factory hooks. [1]

Related v1.0 notes also highlight the httpx backend migration as a major change in the release. [2]

Sources: [1] migration guide, [2] v1.0 announcement blog.

---

🏁 Script executed:

# Check the entire bfcl.py file for huggingface_hub API usage
cat nemo_skills/inference/eval/bfcl.py | grep -n "configure_http_backend\|get_session\|Repository\|InferenceApi\|HfFolder\|hf_transfer"

Repository: NVIDIA-NeMo/Skills

Length of output: 44

---

🏁 Script executed:

# Check bfcl_utils.py for any huggingface_hub usage
grep -n "huggingface_hub\|configure_http_backend\|get_session\|Repository\|InferenceApi" nemo_skills/inference/eval/bfcl_utils.py

Repository: NVIDIA-NeMo/Skills

Length of output: 44

---

🏁 Script executed:

# Look at the entire requirements to understand what's being used
cat nemo_skills/inference/eval/bfcl.py | grep -A 50 "BFCL_REQUIREMENTS = "

Repository: NVIDIA-NeMo/Skills

Length of output: 1667

---

🌐 Web query:

transformers library requirements huggingface_hub version 2025

💡 Result:

In Transformers 4.x releases during 2025, the huggingface_hub requirement (from setup.py) was:

• transformers v4.53.0 (released Jun 26, 2025): huggingface-hub>=0.30.0,<1.0 [1]
• transformers v4.56.0 (released Aug 29, 2025): huggingface-hub>=0.34.0,<1.0 [2]
• transformers v4.57.1 (released Oct 14, 2025): huggingface-hub>=0.34.0,<1.0 [2]

For context, huggingface_hub versions available in 2025 included 0.28.x–0.32.x (and beyond), per PyPI’s release history. [3]

---

🌐 Web query:

gorilla berkeley-function-call-leaderboard huggingface_hub requirements

💡 Result:

In the Gorilla Berkeley Function-Calling Leaderboard (the bfcl_eval / berkeley-function-call-leaderboard package), the dependency list in pyproject.toml includes huggingface_hub with no pinned minimum/maximum version (it’s listed simply as huggingface_hub). [1]

If you’re missing it, install it via your BFCL install (e.g., pip install -e .) or directly with pip install huggingface_hub. [1][2]

---

Drop the unbounded huggingface_hub constraint; transformers still requires huggingface_hub<1.0.

The review comment's description of huggingface_hub v1.0 breaking changes (released Oct 27, 2025) is accurate: the library migrated from requests to httpx, removed configure_http_backend(), changed HTTP error inheritance, and requires Python 3.9+. However, the core issue is not about internal BFCL compatibility—it's about dependency conflict: all current transformers library versions (4.53, 4.56, 4.57.1 from 2025) explicitly pin huggingface-hub<1.0. Since bfcl.py line 24 imports AutoTokenizer from transformers, removing the upper bound on huggingface_hub creates an unresolvable version constraint conflict that will break installation or cause pip to silently downgrade one of the two.

Keep the constraint as huggingface_hub<2 to maintain compatibility with transformers, or upgrade to a version of transformers that supports huggingface_hub>=1.0 before making this change.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@nemo_skills/inference/eval/bfcl.py` at line 57, The dependency change removed
the upper bound on huggingface_hub which conflicts with the transformers package
imported (AutoTokenizer) in bfcl.py; restore a safe upper bound (e.g., set
huggingface_hub<2 in your dependency listing) to avoid pip resolution conflicts
with current transformers versions, or if you truly need huggingface_hub>=1.0
first upgrade transformers to a release that supports it before removing the
bound; locate the dependency entry that was changed (the "huggingface_hub" line)
and revert/update it accordingly, referencing the AutoTokenizer import in
bfcl.py as the reason for preserving compatibility.

[Jorjeous]

LGTM, is this leftover set?