Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 0 additions & 16 deletions .github/workflows/python-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,6 @@ jobs:
test_name: "fastai"
- python-version: "3.10" # fastai not supported on 3.12 and 3.11 -> test it on 3.10
test_name: "fastai"
- python-version: "3.9"
test_name: "tensorflow"
- python-version: "3.10" # tensorflow not supported on 3.12 -> test it on 3.10
test_name: "tensorflow"
- python-version: "3.9" # test torch~=1.11 on python 3.9 only.
test_name: "Python 3.9, torch_1.11"
- python-version: "3.12" # test torch latest on python 3.12 only.
Expand Down Expand Up @@ -83,12 +79,6 @@ jobs:
uv pip install torch~=1.11
;;

tensorflow)
sudo apt update
sudo apt install -y graphviz
uv pip install "huggingface_hub[tensorflow-testing] @ ."
;;

esac

# If not "Xet only", we want to test upload/download with regular LFS workflow
Expand Down Expand Up @@ -127,12 +117,6 @@ jobs:
eval "$PYTEST ../tests/test_fastai*"
;;

tensorflow)
# Cannot be on same line since '_tf*' checks if tensorflow is NOT imported by default
eval "$PYTEST ../tests/test_tf*"
eval "$PYTEST ../tests/test_serialization.py"
;;

"Python 3.9, torch_1.11" | torch_latest)
eval "$PYTEST ../tests/test_hub_mixin*"
eval "$PYTEST ../tests/test_serialization.py"
Expand Down
8 changes: 2 additions & 6 deletions docs/source/cn/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,19 +48,15 @@ pip install --upgrade huggingface_hub
您可以通过`pip`安装可选依赖项,请运行以下代码:

```bash
# 安装 TensorFlow 特定功能的依赖项
# /!\ 注意:这不等同于 `pip install tensorflow`
pip install 'huggingface_hub[tensorflow]'

# 安装 TensorFlow 特定功能和 CLI 特定功能的依赖项
# 安装 Torch 特定功能和 CLI 特定功能的依赖项
pip install 'huggingface_hub[cli,torch]'
```

这里列出了 `huggingface_hub` 的可选依赖项:

- `cli`:为 `huggingface_hub` 提供更方便的命令行界面

- `fastai`,` torch`, `tensorflow`: 运行框架特定功能所需的依赖项
- `fastai`,` torch`: 运行框架特定功能所需的依赖项

- `dev`:用于为库做贡献的依赖项。包括 `testing`(用于运行测试)、`typing`(用于运行类型检查器)和 `quality`(用于运行 linter)

Expand Down
6 changes: 1 addition & 5 deletions docs/source/de/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,18 +44,14 @@ Einige Abhängigkeiten von `huggingface_hub` sind [optional](https://setuptools.

Sie können optionale Abhängigkeiten über `pip` installieren:
```bash
# Abhängigkeiten für spezifische TensorFlow-Funktionen installieren
# /!\ Achtung: dies entspricht nicht `pip install tensorflow`
pip install 'huggingface_hub[tensorflow]'

# Abhängigkeiten sowohl für torch-spezifische als auch für CLI-spezifische Funktionen installieren.
pip install 'huggingface_hub[cli,torch]'
```

Hier ist die Liste der optionalen Abhängigkeiten in huggingface_hub:

- `cli`: bietet eine komfortablere CLI-Schnittstelle für huggingface_hub.
- `fastai`, `torch`, `tensorflow`: Abhängigkeiten, um framework-spezifische Funktionen auszuführen.
- `fastai`, `torch`: Abhängigkeiten, um framework-spezifische Funktionen auszuführen.
- `dev`: Abhängigkeiten, um zur Bibliothek beizutragen. Enthält `testing` (um Tests auszuführen), `typing` (um den Type Checker auszuführen) und `quality` (um Linters auszuführen).


Expand Down
1 change: 0 additions & 1 deletion docs/source/en/guides/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -576,7 +576,6 @@ Copy-and-paste the text below in your GitHub issue.
- Who am I ?: Wauplin
- Configured git credential helpers: store
- FastAI: N/A
- Tensorflow: 2.11.0
- Torch: 1.12.1
- Jinja2: 3.1.2
- Graphviz: 0.20.1
Expand Down
6 changes: 1 addition & 5 deletions docs/source/en/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,17 +46,13 @@ Some dependencies of `huggingface_hub` are [optional](https://setuptools.pypa.io

You can install optional dependencies via `pip`:
```bash
# Install dependencies for tensorflow-specific features
# /!\ Warning: this is not equivalent to `pip install tensorflow`
pip install 'huggingface_hub[tensorflow]'

# Install dependencies for both torch-specific and CLI-specific features.
pip install 'huggingface_hub[cli,torch]'
```

Here is the list of optional dependencies in `huggingface_hub`:
- `cli`: provide a more convenient CLI interface for `huggingface_hub`.
- `fastai`, `torch`, `tensorflow`: dependencies to run framework-specific features.
- `fastai`, `torch`: dependencies to run framework-specific features.
- `dev`: dependencies to contribute to the lib. Includes `testing` (to run tests), `typing` (to run type checker) and `quality` (to run linters).


Expand Down
7 changes: 1 addition & 6 deletions docs/source/en/package_reference/serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,11 +131,7 @@ If you want to save a state dictionary (e.g. a mapping between layer names and r
[[autodoc]] huggingface_hub.save_torch_state_dict


The `serialization` module also contains low-level helpers to split a state dictionary into several shards, while creating a proper index in the process. These helpers are available for `torch` and `tensorflow` tensors and are designed to be easily extended to any other ML frameworks.

### split_tf_state_dict_into_shards

[[autodoc]] huggingface_hub.split_tf_state_dict_into_shards
The `serialization` module also contains low-level helpers to split a state dictionary into several shards, while creating a proper index in the process. These helpers are available for `torch` tensors and are designed to be easily extended to any other ML frameworks.

### split_torch_state_dict_into_shards

Expand All @@ -159,7 +155,6 @@ The loading helpers support both single-file and sharded checkpoints in either s

[[autodoc]] huggingface_hub.load_state_dict_from_file


## Tensors helpers

### get_torch_storage_id
Expand Down
6 changes: 1 addition & 5 deletions docs/source/fr/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,17 +48,13 @@ Toutefois, certaines fonctionnalités de `huggingface_hub` ne seront pas disponi

Vous pouvez installer des dépendances optionnelles via `pip`:
```bash
#Installation des dépendances pour les fonctionnalités spécifiques à Tensorflow.
#/!\ Attention : cette commande n'est pas équivalente à `pip install tensorflow`.
pip install 'huggingface_hub[tensorflow]'

#Installation des dépendances spécifiques à Pytorch et au CLI.
pip install 'huggingface_hub[cli,torch]'
```

Voici une liste des dépendances optionnelles dans `huggingface_hub`:
- `cli` fournit une interface d'invite de commande plus pratique pour `huggingface_hub`.
- `fastai`, `torch` et `tensorflow` sont des dépendances pour utiliser des fonctionnalités spécifiques à un framework.
- `fastai`, `torch` sont des dépendances pour utiliser des fonctionnalités spécifiques à un framework.
- `dev` permet de contribuer à la librairie. Cette dépendance inclut `testing` (pour lancer des tests), `typing` (pour lancer le vérifieur de type) et `quality` (pour lancer des linters).


Expand Down
6 changes: 1 addition & 5 deletions docs/source/hi/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,17 +46,13 @@ pip install --upgrade huggingface_hub

आप `pip` के माध्यम से वैकल्पिक निर्भरताएँ स्थापित कर सकते हैं:
```bash
# Install dependencies for tensorflow-specific features
# /!\ Warning: this is not equivalent to `pip install tensorflow`
pip install 'huggingface_hub[tensorflow]'

# Install dependencies for both torch-specific and CLI-specific features.
pip install 'huggingface_hub[cli,torch]'
```

यहां `huggingface_hub` में वैकल्पिक निर्भरताओं की सूची दी गई है:
- `cli`: `huggingface_hub` के लिए अधिक सुविधाजनक CLI इंटरफ़ेस प्रदान करें।
- `fastai`, `torch`, `tensorflow`: फ्रेमवर्क-विशिष्ट सुविधाओं को चलाने के लिए निर्भरताएँ।
- `fastai`, `torch`: फ्रेमवर्क-विशिष्ट सुविधाओं को चलाने के लिए निर्भरताएँ।
- `dev`: lib में योगदान करने के लिए निर्भरताएँ। इसमें 'परीक्षण' (परीक्षण चलाने के लिए), 'टाइपिंग' (टाइप चेकर चलाने के लिए) और 'गुणवत्ता' (लिंटर चलाने के लिए) शामिल हैं।


Expand Down
1 change: 0 additions & 1 deletion docs/source/ko/guides/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -455,7 +455,6 @@ Copy-and-paste the text below in your GitHub issue.
- Who am I ?: Wauplin
- Configured git credential helpers: store
- FastAI: N/A
- Tensorflow: 2.11.0
- Torch: 1.12.1
- Jinja2: 3.1.2
- Graphviz: 0.20.1
Expand Down
6 changes: 1 addition & 5 deletions docs/source/ko/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,17 +46,13 @@ pip install --upgrade huggingface_hub

선택적 의존성은 `pip`을 통해 설치할 수 있습니다:
```bash
# TensorFlow 관련 기능에 대한 의존성을 설치합니다.
# /!\ 경고: `pip install tensorflow`와 동일하지 않습니다.
pip install 'huggingface_hub[tensorflow]'

# PyTorch와 CLI와 관련된 기능에 대한 의존성을 모두 설치합니다.
pip install 'huggingface_hub[cli,torch]'
```

다음은 `huggingface_hub`의 선택 의존성 목록입니다:
- `cli`: 보다 편리한 `huggingface_hub`의 CLI 인터페이스입니다.
- `fastai`, `torch`, `tensorflow`: 프레임워크별 기능을 실행하려면 필요합니다.
- `fastai`, `torch`: 프레임워크별 기능을 실행하려면 필요합니다.
- `dev`: 라이브러리에 기여하고 싶다면 필요합니다. 테스트 실행을 위한 `testing`, 타입 검사기 실행을 위한 `typing`, 린터 실행을 위한 `quality`가 포함됩니다.

### 소스에서 설치 [[install-from-source]]
Expand Down
6 changes: 1 addition & 5 deletions docs/source/ko/package_reference/serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,7 @@ rendered properly in your Markdown viewer.

## 상태 사전을 샤드로 나누기[[split-state-dict-into-shards]]

현재 이 모듈은 상태 딕셔너리(예: 레이어 이름과 관련 텐서 간의 매핑)를 받아 여러 샤드로 나누고, 이 과정에서 적절한 인덱스를 생성하는 단일 헬퍼를 포함하고 있습니다. 이 헬퍼는 `torch`, `tensorflow`, `numpy` 텐서에 사용 가능하며, 다른 ML 프레임워크로 쉽게 확장될 수 있도록 설계되었습니다.

### split_tf_state_dict_into_shards[[huggingface_hub.split_tf_state_dict_into_shards]]

[[autodoc]] huggingface_hub.split_tf_state_dict_into_shards
현재 이 모듈은 상태 딕셔너리(예: 레이어 이름과 관련 텐서 간의 매핑)를 받아 여러 샤드로 나누고, 이 과정에서 적절한 인덱스를 생성하는 단일 헬퍼를 포함하고 있습니다. 이 헬퍼는 `torch` 텐서에 사용 가능하며, 다른 ML 프레임워크로 쉽게 확장될 수 있도록 설계되었습니다.

### split_torch_state_dict_into_shards[[huggingface_hub.split_torch_state_dict_into_shards]]

Expand Down
6 changes: 1 addition & 5 deletions docs/source/tm/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,17 +43,13 @@ pip install --upgrade huggingface_hub
நீங்கள் விருப்பத் தேவைப்படும் சார்புகளை `pip` மூலம் நிறுவலாம்:

```bash
# டென்சர்‌ஃபிளோவுக்கான குறிப்பிட்ட அம்சங்களுக்கு சார்ந்த பொறுப்பு நிறுவவும்
# /!\ எச்சரிக்கை: இது `pip install tensorflow` க்கு சமமாகக் கருதப்படாது
pip install 'huggingface_hub[tensorflow]'

# டார்ச்-குறிப்பிட்ட மற்றும் CLI-குறிப்பிட்ட அம்சங்களுக்கு தேவையான பொறுப்புகளை நிறுவவும்.
pip install 'huggingface_hub[cli,torch]'
```
`huggingface_hub`-இல் உள்ள விருப்பத் தேவைப்படும் சார்புகளின் பட்டியல்:

- `cli`: `huggingface_hub`-க்கு மிகவும் வசதியான CLI இடைமுகத்தை வழங்குகிறது.
- `fastai`, `torch`, `tensorflow`: வடிவமைப்பு குறிப்பிட்ட அம்சங்களை இயக்க தேவையான சார்புகள்.
- `fastai`, `torch`: வடிவமைப்பு குறிப்பிட்ட அம்சங்களை இயக்க தேவையான சார்புகள்.
- `dev`: நூலகத்திற்கு பங்களிக்க தேவையான சார்புகள். இதில் சோதனை (சோதனைகளை இயக்க), வகை சோதனை (வகை சரிபார்ப்பு ஐ இயக்க) மற்றும் தரம் (லிண்டர்கள் ஐ இயக்க) உள்ளன.

### மூலத்திலிருந்து நிறுவல்
Expand Down
10 changes: 0 additions & 10 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,6 @@ def get_version() -> str:
"fastcore>=1.3.27",
]

extras["tensorflow"] = [
"tensorflow",
"pydot",
"graphviz",
]

extras["tensorflow-testing"] = [
"tensorflow",
]

extras["hf_xet"] = ["hf-xet>=1.1.2,<2.0.0"]

extras["mcp"] = [
Expand Down
6 changes: 0 additions & 6 deletions src/huggingface_hub/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -488,15 +488,13 @@
],
"serialization": [
"StateDictSplit",
"get_tf_storage_size",
"get_torch_storage_id",
"get_torch_storage_size",
"load_state_dict_from_file",
"load_torch_model",
"save_torch_model",
"save_torch_state_dict",
"split_state_dict_into_shards_factory",
"split_tf_state_dict_into_shards",
"split_torch_state_dict_into_shards",
],
"serialization._dduf": [
Expand Down Expand Up @@ -869,7 +867,6 @@
"get_session",
"get_space_runtime",
"get_space_variables",
"get_tf_storage_size",
"get_token",
"get_torch_storage_id",
"get_torch_storage_size",
Expand Down Expand Up @@ -950,7 +947,6 @@
"snapshot_download",
"space_info",
"split_state_dict_into_shards_factory",
"split_tf_state_dict_into_shards",
"split_torch_state_dict_into_shards",
"super_squash_history",
"suspend_scheduled_job",
Expand Down Expand Up @@ -1494,15 +1490,13 @@ def __dir__():
)
from .serialization import (
StateDictSplit, # noqa: F401
get_tf_storage_size, # noqa: F401
get_torch_storage_id, # noqa: F401
get_torch_storage_size, # noqa: F401
load_state_dict_from_file, # noqa: F401
load_torch_model, # noqa: F401
save_torch_model, # noqa: F401
save_torch_state_dict, # noqa: F401
split_state_dict_into_shards_factory, # noqa: F401
split_tf_state_dict_into_shards, # noqa: F401
split_torch_state_dict_into_shards, # noqa: F401
)
from .serialization._dduf import (
Expand Down
1 change: 0 additions & 1 deletion src/huggingface_hub/serialization/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
"""Contains helpers to serialize tensors."""

from ._base import StateDictSplit, split_state_dict_into_shards_factory
from ._tensorflow import get_tf_storage_size, split_tf_state_dict_into_shards
from ._torch import (
get_torch_storage_id,
get_torch_storage_size,
Expand Down
95 changes: 0 additions & 95 deletions src/huggingface_hub/serialization/_tensorflow.py

This file was deleted.

Loading
Loading