Skip to content

Commit c71f360

Browse files
committed
Install tfds-nightly for nightly testing
While debugging our latest nightly failures, I noticed we were downloading a ton of stable tf versions into our nightly image (and ultimately installing one). Switching the nightly requirements file to pull the nightly version of tfds should avoid this.
1 parent 00fe9a1 commit c71f360

File tree

6 files changed

+11
-12
lines changed

6 files changed

+11
-12
lines changed

examples/bert_pretraining/README.md

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -39,13 +39,12 @@ python3 examples/bert_pretraining/bert_pretrain.py \
3939

4040
## Installing dependencies
4141

42-
Pip dependencies for all KerasNLP examples are listed in `setup.py`. The
43-
following command will create a virtual environment, install all dependencies,
44-
and install KerasNLP from source.
42+
This example needs with a few extra dependecies to run (e.g. wikiextractor for
43+
using wikipedia downloads). You can install these into a KerasNLP development
44+
environment with:
4545

4646
```shell
47-
python3 -m venv path/to/venv && source path/to/venv/bin/activate
48-
pip install -e ".[examples]"
47+
pip install -r "examples/bert_pretraining/requirements.txt"
4948
```
5049

5150
## Pretraining BERT
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
nltk
2+
wikiextractor

requirements-common.txt

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,3 @@ pytest-cov
1111
# Optional deps.
1212
rouge-score
1313
sentencepiece
14-
# Examples deps.
15-
nltk
16-
tensorflow_datasets
17-
wikiextractor
18-
keras-tuner

requirements-macos-m1.txt

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,11 @@
66
# before proceeding.
77

88
# Core deps.
9-
tensorflow-macos==2.9
9+
tensorflow-macos~=2.9
1010
https://github.com/sun1638650145/Libraries-and-Extensions-for-TensorFlow-for-Apple-Silicon/releases/download/v2.9/tensorflow_text-2.9.0-cp39-cp39-macosx_11_0_arm64.whl
11+
tensorflow-datasets~=2.9
1112
# The metal plugin breaks many tests, so is not enabled by default.
12-
# tensorflow-metal==0.5.1
13+
# tensorflow-metal~=0.5
1314

1415
# Common deps.
1516
-r requirements-common.txt

requirements-nightly.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Core deps.
22
tf-nightly
33
tensorflow-text-nightly
4+
tfds-nightly
45

56
# Common deps.
67
-r requirements-common.txt

requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Core deps.
22
tensorflow~=2.11.0
33
tensorflow-text~=2.11.0
4+
tensorflow-datasets~=2.11.0
45

56
# Common deps.
67
-r requirements-common.txt

0 commit comments

Comments
 (0)