Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs m1 #11578

Merged
merged 27 commits into from
Oct 7, 2022
Merged

Docs m1 #11578

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
f5b211e
reverted docker build changes, removing aarch64 builds
tmbo Sep 19, 2022
94c25a7
added documentation for M1 installations
tmbo Sep 19, 2022
fa29f47
Update docs/docs/installation.mdx
tmbo Sep 19, 2022
48d3c9d
Apply suggestions from code review
tmbo Sep 20, 2022
0549b64
Update docs/docs/components.mdx
tmbo Sep 20, 2022
8af68d8
removed unneded type ignore that let ci fail
tmbo Sep 21, 2022
eb1d687
Merge branch 'docs-m1' of github.com:RasaHQ/rasa into docs-m1
tmbo Sep 21, 2022
7dd0f90
bumped spacy to 3.4
twerkmeister Sep 26, 2022
6459f18
explicitly defined dependency for spacy featurizer
twerkmeister Sep 26, 2022
2d37a99
added external dependency versions as a fingerprint attribute for com…
twerkmeister Sep 26, 2022
b1fa276
Applying default config in SpacyNLP component
twerkmeister Sep 26, 2022
3fd2184
Removed fixed vector testing, improved case test
twerkmeister Sep 26, 2022
c5b4d47
Reformatted using black
twerkmeister Sep 26, 2022
fbd5aa3
Added changelog
twerkmeister Sep 26, 2022
0d6a144
Exchanged importlib with pkg_resources for py37 compatibility
twerkmeister Sep 26, 2022
1b1fe4d
using direct import
twerkmeister Sep 26, 2022
b3efbdf
Back to pkg-resources but with mapping for sklearn
twerkmeister Sep 27, 2022
41ae795
Unified definitions of required_packages as staticmethod in all compo…
twerkmeister Sep 27, 2022
bfbc0a3
code quality
twerkmeister Sep 27, 2022
1ac7fe7
made tensorflow metal optional
twerkmeister Oct 4, 2022
5b653cd
Added note about retraining to changelog
twerkmeister Oct 4, 2022
622521f
Distinguishing between m1 and others for spacy dependency
twerkmeister Oct 4, 2022
2a92712
merged spacy 3.4 update
tmbo Oct 5, 2022
99ff6f9
merged main
tmbo Oct 5, 2022
162bd1a
fixed testing dependency issue
tmbo Oct 7, 2022
9aa8e76
fixed invalid docs links
tmbo Oct 7, 2022
1deaf4b
Merge branch 'main' into docs-m1
tmbo Oct 7, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 7 additions & 21 deletions .github/workflows/continous-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -552,9 +552,6 @@ jobs:
uses: actions/checkout@v3
if: github.event_name != 'pull_request'

- name: Set up QEMU
uses: docker/setup-qemu-action@v1

- name: Read Poetry Version 🔢
run: |
echo "POETRY_VERSION=$(scripts/poetry-version.sh)" >> $GITHUB_ENV
Expand Down Expand Up @@ -624,30 +621,26 @@ jobs:
if: steps.check_image.outputs.base_exists == 'false' || env.IS_TAG_BUILD == 'true'
run: |
export IMAGE_TAG=${{ steps.check_image.outputs.base_image_hash }}
docker buildx bake --set *.platform=linux/amd64 -f docker/docker-bake.hcl base --push
docker buildx bake --set *.platform=linux/arm64 -f docker/docker-bake.hcl base --push
docker buildx bake -f docker/docker-bake.hcl base --push

- name: Build Docker mitie base image and push 🛠 ⬆
if: steps.check_image.outputs.base_mitie_exists == 'false'
run: |
export IMAGE_TAG=${{ steps.check_image.outputs.base_mitie_image_hash }}
docker buildx bake --set *.platform=linux/amd64 -f docker/docker-bake.hcl base-mitie --push
docker buildx bake --set *.platform=linux/arm64 -f docker/docker-bake.hcl base-mitie --push
docker buildx bake -f docker/docker-bake.hcl base-mitie --push

- name: Build Docker poetry base image and push 🛠 ⬆
if: steps.check_image.outputs.base_poetry_exists == 'false'
run: |
export IMAGE_TAG=${{ env.POETRY_VERSION }}
export BASE_IMAGE_HASH=${{ steps.check_image.outputs.base_image_hash }}
docker buildx bake --set *.platform=linux/amd64 -f docker/docker-bake.hcl base-poetry --push
docker buildx bake --set *.platform=linux/arm64 -f docker/docker-bake.hcl base-poetry --push
docker buildx bake -f docker/docker-bake.hcl base-poetry --push

- name: Build Docker builder base image and push 🛠 ⬆
if: steps.check_image.outputs.base_builder_exists == 'false'
run: |
export IMAGE_TAG=${{ steps.check_image.outputs.base_builder_image_hash }}
docker buildx bake --set *.platform=linux/amd64 -f docker/docker-bake.hcl base-builder --push
docker buildx bake --set *.platform=linux/arm64 -f docker/docker-bake.hcl base-builder --push
docker buildx bake -f docker/docker-bake.hcl base-builder --push

# Set environment variables for a pull request
#
Expand Down Expand Up @@ -704,7 +697,6 @@ jobs:
strategy:
matrix:
image: [default, full, mitie-en, spacy-de, spacy-en]
arch: [linux/amd64, linux/arm64]

steps:
# Due to an issue with checking out a wrong commit, we make sure
Expand All @@ -720,9 +712,6 @@ jobs:
uses: actions/checkout@v3
if: github.event_name != 'pull_request'

- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Free disk space
if: needs.changes.outputs.docker == 'true'
# tries to make sure we do not run out of disk space, see
Expand All @@ -746,9 +735,6 @@ jobs:
version: v0.5.1
driver: docker

- name: Echo Available platforms
run: echo ${{ steps.buildx.outputs.platforms }}

- name: Login to DockerHub Registry 🔢
if: needs.changes.outputs.docker == 'true'
run: echo ${{ secrets.DOCKERHUB_PASSWORD }} | docker login -u ${{ env.DOCKERHUB_USERNAME }} --password-stdin || true
Expand All @@ -764,19 +750,19 @@ jobs:
- name: Build Docker image
if: needs.changes.outputs.docker == 'true'
run: |
docker buildx bake --set *.platform=${{ matrix.arch }} -f docker/docker-bake.hcl ${{ matrix.image }}
docker buildx bake -f docker/docker-bake.hcl ${{ matrix.image }}

- name: Push image with main tag 📦
if: needs.changes.outputs.docker == 'true' && github.event_name == 'push' && github.ref == 'refs/heads/main' && github.repository == 'RasaHQ/rasa'
run: |
docker buildx bake --set *.platform=${{ matrix.arch }} -f docker/docker-bake.hcl ${{ matrix.image }} --push
docker buildx bake -f docker/docker-bake.hcl ${{ matrix.image }} --push

- name: Push image with ${{github.ref}} tag 📦
if: needs.changes.outputs.docker == 'true' && github.event_name == 'push' && env.IS_TAG_BUILD == 'true' && github.repository == 'RasaHQ/rasa'
run: |
IS_NEWEST_VERSION=${{ needs.build_docker_base_images_and_set_env.outputs.is_newest_version }}

docker buildx bake --set *.platform=${{ matrix.arch }} -f docker/docker-bake.hcl ${{ matrix.image }} --push
docker buildx bake -f docker/docker-bake.hcl ${{ matrix.image }} --push

# Tag the image as latest
if [[ "${IS_NEWEST_VERSION}" == "true" ]]; then
Expand Down
2 changes: 2 additions & 0 deletions changelog/ATO-320.improvement.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Rasa supports native installations on Apple Silicon (M1 / M2). Please
follow the installation instructions and take a look at the limitations.
2 changes: 2 additions & 0 deletions docs/docs/components.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -495,6 +495,8 @@ Note: The `feature-dimension` for sequence and sentence features does not have t
:::note
To use `ConveRTFeaturizer`, install Rasa Open Source with `pip3 install rasa[convert]`.

Note that this component cannot currently run on MacOS using M1 / M2 architecture.
More information on this limitation is available [here](./installation.mdx#m1--m2-apple-silicon-limitations).
:::


Expand Down
37 changes: 33 additions & 4 deletions docs/docs/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,17 @@ your machine and continue development locally.
</TabItem>
<TabItem value="macos">

:::info

Rasa supports **x86_64 (Intel) and M1 / M2 (Apple Silicon)** architectures.
The support is for development purposes only and we strongly recommend
production deployments to use Ubuntu / Linux on x86_64.

More information about the limitations of rasa installations on Apple Silicon can be found
in this [section](#m1--m2-apple-silicon-limitations).

:::

Create a new virtual environment by choosing a Python interpreter and making a `./venv` directory to hold it:

```bash
Expand Down Expand Up @@ -86,10 +97,6 @@ rasa init

You can learn about the most important Rasa commands in the [Command Line Interface](./command-line-interface.mdx).

:::note
Due to the lack of official TensorFlow support for the Apple M1, Rasa Open Source is currently unable to train a model using M1.
:::

## Step-by-step Installation Guide

Prefer following video instructions? Watch our installation series on
Expand Down Expand Up @@ -316,3 +323,25 @@ To download a specific version, specify the version number:
```bash
pip3 install rasa==3.0
```

## M1 / M2 (Apple Silicon) Limitations

Rasa installations on Apple Silicon use [Apple Metal](https://developer.apple.com/metal/) leading to reduced training times
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make metal optional, and change this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mind doing the change directly on this branch?

for local development.

Currently, not all of Rasa's dependencies support Apple Silicon natively. This leads
to the following restrictions:

* You can not run Duckling as a docker container on Apple Silicon. If you want
to use the [duckling entity extractor](components.mdx#ducklingentityextractor) we recommend a cloud deployment of
duckling. Progress on this can be tracked on the
[Duckling project](https://github.com/facebook/duckling/issues/695).
* You cannot run Rasa in a Docker container on Apple Silicon, only native installations
currently work. An installation in docker requires support for Ubuntu aarch64
which the current tensorflow version 2.8 does not provide - only MacOS is
supported as an operating system running on aarch64. We expect a future
upgrade of Tensorflow to allow Apple Silicon users to run Rasa inside of Docker.
* Rasa on Apple Silicon does not support the [`ConveRTFeaturizer` component](components.mdx#convertfeaturizer) or pipelines
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also spacy isn't supported until we upgrade to 3.4.1.

containing it. The component relies on `tensorflow-text` which currently
isn't available for Apple Silicon. Progress on this can be tracked on the
[Tensorflow Text project](https://github.com/tensorflow/text/issues/823).
Loading