Skip to content

Commit

Permalink
Remove Tensorflow from default Beam images. (#28424)
Browse files Browse the repository at this point in the history
  • Loading branch information
@tvalentyn
tvalentyn authored Sep 12, 2023
1 parent cb05bd1 commit 9f3bea9
Show file tree
Hide file tree
Showing 6 changed files with 32 additions and 125 deletions.
2 changes: 1 addition & 1 deletion CHANGES.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,8 @@

## Breaking Changes

* X behavior was changed ([#X](https://github.com/apache/beam/issues/X)).
* Removed fastjson library dependency for Beam SQL. Table property is changed to be based on jackson ObjectNode (Java) ([#24154](https://github.com/apache/beam/issues/24154)).
* Removed TensorFlow from Beam Python container images [PR](https://github.com/apache/beam/pull/28424). If you have been negatively affected by this change, please comment on [#20605](https://github.com/apache/beam/issues/20605).

## Deprecations

Expand Down
1 change: 0 additions & 1 deletion sdks/python/container/base_image_requirements_manual.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,4 +43,3 @@ nose==1.3.7 # For Dataflow internal testing. TODO: remove this.
python-snappy;python_version<"3.11" # Optimizes execution of some Beam codepaths.
scipy
scikit-learn
tensorflow>=2.12.0
38 changes: 8 additions & 30 deletions sdks/python/container/py310/base_image_requirements.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@
# https://s.apache.org/beam-python-dev-wiki
# Reach out to a committer if you need help.

absl-py==1.4.0
astunparse==1.6.3
attrs==23.1.0
beautifulsoup4==4.12.2
bs4==0.0.1
Expand All @@ -44,16 +42,13 @@ exceptiongroup==1.1.3
execnet==2.0.2
fastavro==1.8.3
fasteners==0.18
flatbuffers==23.5.26
freezegun==1.2.2
future==0.18.3
gast==0.4.0
google-api-core==2.11.1
google-api-python-client==2.98.0
google-api-python-client==2.99.0
google-apitools==0.5.31
google-auth==2.22.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-auth==2.23.0
google-auth-httplib2==0.1.1
google-cloud-aiplatform==1.32.0
google-cloud-bigquery==3.11.4
google-cloud-bigquery-storage==2.22.0
Expand All @@ -63,7 +58,7 @@ google-cloud-datastore==2.18.0
google-cloud-dlp==3.12.2
google-cloud-language==2.11.0
google-cloud-profiler==4.1.0
google-cloud-pubsub==2.18.3
google-cloud-pubsub==2.18.4
google-cloud-pubsublite==1.8.3
google-cloud-recommendations-ai==0.10.4
google-cloud-resource-manager==1.10.3
Expand All @@ -72,34 +67,26 @@ google-cloud-storage==2.10.0
google-cloud-videointelligence==2.11.3
google-cloud-vision==3.4.4
google-crc32c==1.5.0
google-pasta==0.2.0
google-resumable-media==2.6.0
googleapis-common-protos==1.60.0
greenlet==2.0.2
grpc-google-iam-v1==0.12.6
grpcio==1.58.0
grpcio-status==1.58.0
guppy3==3.1.3
h5py==3.9.0
hdfs==2.7.2
httplib2==0.22.0
hypothesis==6.84.2
hypothesis==6.84.3
idna==3.4
iniconfig==2.0.0
joblib==1.3.2
keras==2.13.1
libclang==16.0.6
Markdown==3.4.4
MarkupSafe==2.1.3
mmh3==4.0.1
mock==5.1.0
nltk==3.8.1
nose==1.3.7
numpy==1.24.3
numpy==1.24.4
oauth2client==4.1.3
oauthlib==3.2.2
objsize==0.6.1
opt-einsum==3.3.0
orjson==3.9.7
overrides==6.5.0
packaging==23.1
Expand Down Expand Up @@ -128,7 +115,6 @@ PyYAML==6.0.1
regex==2023.8.8
requests==2.31.0
requests-mock==1.11.0
requests-oauthlib==1.3.1
rsa==4.9
scikit-learn==1.3.0
scipy==1.11.2
Expand All @@ -139,21 +125,13 @@ soupsieve==2.5
SQLAlchemy==1.4.49
sqlparse==0.4.4
tenacity==8.2.3
tensorboard==2.13.0
tensorboard-data-server==0.7.1
tensorflow==2.13.0
tensorflow-cpu-aws==2.13.0;platform_machine=="aarch64"
tensorflow-estimator==2.13.0
tensorflow-io-gcs-filesystem==0.33.0
termcolor==2.3.0
testcontainers==3.7.1
threadpoolctl==3.2.0
tomli==2.0.1
tqdm==4.66.1
typing_extensions==4.5.0
typing_extensions==4.7.1
uritemplate==4.1.1
urllib3==1.26.16
websocket-client==1.6.2
Werkzeug==2.3.7
websocket-client==1.6.3
wrapt==1.15.0
zstandard==0.21.0
36 changes: 7 additions & 29 deletions sdks/python/container/py311/base_image_requirements.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@
# https://s.apache.org/beam-python-dev-wiki
# Reach out to a committer if you need help.

absl-py==1.4.0
astunparse==1.6.3
attrs==23.1.0
beautifulsoup4==4.12.2
bs4==0.0.1
Expand All @@ -43,15 +41,12 @@ docopt==0.6.2
execnet==2.0.2
fastavro==1.8.3
fasteners==0.18
flatbuffers==23.5.26
freezegun==1.2.2
future==0.18.3
gast==0.4.0
google-api-core==2.11.1
google-apitools==0.5.31
google-auth==2.22.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-auth==2.23.0
google-auth-httplib2==0.1.1
google-cloud-aiplatform==1.32.0
google-cloud-bigquery==3.11.4
google-cloud-bigquery-storage==2.22.0
Expand All @@ -60,7 +55,7 @@ google-cloud-core==2.3.3
google-cloud-datastore==2.18.0
google-cloud-dlp==3.12.2
google-cloud-language==2.11.0
google-cloud-pubsub==2.18.3
google-cloud-pubsub==2.18.4
google-cloud-pubsublite==1.8.3
google-cloud-recommendations-ai==0.10.4
google-cloud-resource-manager==1.10.3
Expand All @@ -69,34 +64,26 @@ google-cloud-storage==2.10.0
google-cloud-videointelligence==2.11.3
google-cloud-vision==3.4.4
google-crc32c==1.5.0
google-pasta==0.2.0
google-resumable-media==2.6.0
googleapis-common-protos==1.60.0
greenlet==2.0.2
grpc-google-iam-v1==0.12.6
grpcio==1.58.0
grpcio-status==1.58.0
guppy3==3.1.3
h5py==3.9.0
hdfs==2.7.2
httplib2==0.22.0
hypothesis==6.84.2
hypothesis==6.84.3
idna==3.4
iniconfig==2.0.0
joblib==1.3.2
keras==2.13.1
libclang==16.0.6
Markdown==3.4.4
MarkupSafe==2.1.3
mmh3==4.0.1
mock==5.1.0
nltk==3.8.1
nose==1.3.7
numpy==1.24.3
numpy==1.24.4
oauth2client==4.1.3
oauthlib==3.2.2
objsize==0.6.1
opt-einsum==3.3.0
orjson==3.9.7
overrides==6.5.0
packaging==23.1
Expand Down Expand Up @@ -124,7 +111,6 @@ PyYAML==6.0.1
regex==2023.8.8
requests==2.31.0
requests-mock==1.11.0
requests-oauthlib==1.3.1
rsa==4.9
scikit-learn==1.3.0
scipy==1.11.2
Expand All @@ -135,19 +121,11 @@ soupsieve==2.5
SQLAlchemy==1.4.49
sqlparse==0.4.4
tenacity==8.2.3
tensorboard==2.13.0
tensorboard-data-server==0.7.1
tensorflow==2.13.0
tensorflow-cpu-aws==2.13.0;platform_machine=="aarch64"
tensorflow-estimator==2.13.0
tensorflow-io-gcs-filesystem==0.33.0
termcolor==2.3.0
testcontainers==3.7.1
threadpoolctl==3.2.0
tqdm==4.66.1
typing_extensions==4.5.0
typing_extensions==4.7.1
urllib3==1.26.16
websocket-client==1.6.2
Werkzeug==2.3.7
websocket-client==1.6.3
wrapt==1.15.0
zstandard==0.21.0
40 changes: 8 additions & 32 deletions sdks/python/container/py38/base_image_requirements.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@
# https://s.apache.org/beam-python-dev-wiki
# Reach out to a committer if you need help.

absl-py==1.4.0
astunparse==1.6.3
attrs==23.1.0
beautifulsoup4==4.12.2
bs4==0.0.1
Expand All @@ -44,16 +42,13 @@ exceptiongroup==1.1.3
execnet==2.0.2
fastavro==1.8.3
fasteners==0.18
flatbuffers==23.5.26
freezegun==1.2.2
future==0.18.3
gast==0.4.0
google-api-core==2.11.1
google-api-python-client==2.98.0
google-api-python-client==2.99.0
google-apitools==0.5.31
google-auth==2.22.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-auth==2.23.0
google-auth-httplib2==0.1.1
google-cloud-aiplatform==1.32.0
google-cloud-bigquery==3.11.4
google-cloud-bigquery-storage==2.22.0
Expand All @@ -63,7 +58,7 @@ google-cloud-datastore==2.18.0
google-cloud-dlp==3.12.2
google-cloud-language==2.11.0
google-cloud-profiler==4.1.0
google-cloud-pubsub==2.18.3
google-cloud-pubsub==2.18.4
google-cloud-pubsublite==1.8.3
google-cloud-recommendations-ai==0.10.4
google-cloud-resource-manager==1.10.3
Expand All @@ -72,35 +67,26 @@ google-cloud-storage==2.10.0
google-cloud-videointelligence==2.11.3
google-cloud-vision==3.4.4
google-crc32c==1.5.0
google-pasta==0.2.0
google-resumable-media==2.6.0
googleapis-common-protos==1.60.0
greenlet==2.0.2
grpc-google-iam-v1==0.12.6
grpcio==1.58.0
grpcio-status==1.58.0
guppy3==3.1.3
h5py==3.9.0
hdfs==2.7.2
httplib2==0.22.0
hypothesis==6.84.2
hypothesis==6.84.3
idna==3.4
importlib-metadata==6.8.0
iniconfig==2.0.0
joblib==1.3.2
keras==2.13.1
libclang==16.0.6
Markdown==3.4.4
MarkupSafe==2.1.3
mmh3==4.0.1
mock==5.1.0
nltk==3.8.1
nose==1.3.7
numpy==1.24.3
numpy==1.24.4
oauth2client==4.1.3
oauthlib==3.2.2
objsize==0.6.1
opt-einsum==3.3.0
orjson==3.9.7
overrides==6.5.0
packaging==23.1
Expand Down Expand Up @@ -129,7 +115,6 @@ PyYAML==6.0.1
regex==2023.8.8
requests==2.31.0
requests-mock==1.11.0
requests-oauthlib==1.3.1
rsa==4.9
scikit-learn==1.3.0
scipy==1.10.1
Expand All @@ -140,22 +125,13 @@ soupsieve==2.5
SQLAlchemy==1.4.49
sqlparse==0.4.4
tenacity==8.2.3
tensorboard==2.13.0
tensorboard-data-server==0.7.1
tensorflow==2.13.0
tensorflow-cpu-aws==2.13.0;platform_machine=="aarch64"
tensorflow-estimator==2.13.0
tensorflow-io-gcs-filesystem==0.33.0
termcolor==2.3.0
testcontainers==3.7.1
threadpoolctl==3.2.0
tomli==2.0.1
tqdm==4.66.1
typing_extensions==4.5.0
typing_extensions==4.7.1
uritemplate==4.1.1
urllib3==1.26.16
websocket-client==1.6.2
Werkzeug==2.3.7
websocket-client==1.6.3
wrapt==1.15.0
zipp==3.16.2
zstandard==0.21.0
Loading

0 comments on commit 9f3bea9

Please sign in to comment.