Skip to content

Commit f156b5e

Browse files
authored
#69: Created slim version of exaslct for MacOsX (#71)
Created a slim version of exaslct which is compatible with restrictions on MacOsX: 1. Using greadlink instead of readlink 2. Not using any newer bash features (associative arrays) due to very old bash version on MacOsX. This prohibits the parsing of the mount binding directories, and restricts the usage of exaslct to directories within the current directory.
1 parent 8c15428 commit f156b5e

17 files changed

+206
-21
lines changed

README.md

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -19,23 +19,32 @@ the following prerequisites:
1919

2020
* Software
2121
* Linux
22-
* [bash](https://www.gnu.org/software/bash/) >= 4.2
22+
* [bash](https://www.gnu.org/software/bash/) >= 4.2
23+
* MacOsX
24+
* [bash](https://www.gnu.org/software/bash/) > 3.2
2325
* [coreutils](https://www.gnu.org/software/coreutils/)
2426
* sha512sum
2527
* sed
2628
* [curl](https://curl.se/)
2729

30+
2831
#### For running
2932

3033
In order to use this tool, your system needs to fulfill the following prerequisites:
3134

3235
* Software
3336
* Linux
34-
* [bash](https://www.gnu.org/software/bash/) >= 4.2
35-
* [coreutils](https://www.gnu.org/software/coreutils/)
36-
* readlink with -f option (the readlink version of macOS doesn't support -f)
37-
* realpath
38-
* dirname
37+
* [bash](https://www.gnu.org/software/bash/) >= 4.2
38+
* [coreutils](https://www.gnu.org/software/coreutils/)
39+
* readlink with -f option
40+
* realpath
41+
* dirname
42+
* MacOsX (Please see limitations on [MacOsX](#macosx-limitations))
43+
* [bash](https://www.gnu.org/software/bash/) >= 3.2
44+
* [coreutils](https://www.gnu.org/software/coreutils/)
45+
* greadlink with -f option
46+
* realpath
47+
* dirname
3948
* [Docker](https://docs.docker.com/) >= 17.05
4049
* with support for [multi-stage builds required](https://docs.docker.com/develop/develop-images/multistage-build/)
4150
* host volume mounts need to be allowed
@@ -140,7 +149,11 @@ ALTER SESSION SET SCRIPT_LANGUAGES='<LANGUAGE_ALIAS>=localzmq+protobuf:///<bucke
140149
We currently support only the mounting of the given command line arguments, but we do not analyze
141150
the content of those directories.
142151
Plan is to fix this limitation with [#35](https://github.com/exasol/script-languages-container-tool/issues/35)
152+
153+
### MacOsX Limitations
143154

155+
* On MacOsX all arguments (flavors path, output directory, etc.) must point to locations within the current directory (background is that the MacOsX version does not support mount binding additional directories).
156+
144157
## Table of Contents
145158

146159
### Information for Users

exaslct

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,22 @@
11
#!/usr/bin/env bash
22

3+
#####################################################################################
4+
###REMEMBER TO TEST ANY CHANGES HERE ON MACOSX!!!
5+
#####################################################################################
6+
37
set -euo pipefail
48

5-
SCRIPT_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
9+
rl=readlink
10+
if [[ "$(uname)" = Darwin ]]; then
11+
rl=greadlink
12+
fi
13+
14+
if [[ ! "$(command -v $rl)" ]]; then
15+
echo readlink not available! Please install coreutils: On Linux \"apt-get install coreutils\" or similar. On MacOsX \"brew install coreutils\".
16+
exit 1
17+
fi
18+
19+
SCRIPT_DIR="$(dirname "$($rl -f "${BASH_SOURCE[0]}")")"
620

721
RUNNER_IMAGE_NAME="$("$SCRIPT_DIR/starter_scripts/construct_docker_runner_image_name.sh")"
822

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
6df6fa47fb39de40428f76f61da10b37099c421c39dba3fe95b85bef8ba651da5762cb3813b6f3a393229b7d6a56257c3148749afb88bfd10d02b7358f278321 exaslct_installer.sh
1+
5d07ac5365bbcbb60c6de07ffa9f2b27fcdea3c66782f3681061c54005b2dca637efb4964f5b657504dbfbf68ed79bcd911d6e1e30a5d24157ebd08f3f97988f exaslct_installer.sh

installer/exaslct_installer.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ main() {
6666
#Take care and make sure list of files is identical!!!
6767
download_and_verify_raw_file_from_github "$repo" "$exaslct_git_ref" "starter_scripts/exaslct_within_docker_container_without_container_build.sh"
6868
download_and_verify_raw_file_from_github "$repo" "$exaslct_git_ref" "starter_scripts/exaslct_within_docker_container.sh"
69+
download_and_verify_raw_file_from_github "$repo" "$exaslct_git_ref" "starter_scripts/exaslct_within_docker_container_slim.sh"
6970
download_and_verify_raw_file_from_github "$repo" "$exaslct_git_ref" "starter_scripts/construct_docker_runner_image_name.sh"
7071
download_and_verify_raw_file_from_github "$repo" "$exaslct_git_ref" "starter_scripts/mount_point_parsing.sh"
7172
download_and_verify_raw_file_from_github "$repo" "$exaslct_git_ref" "installer/exaslct_install_template.sh"

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
'name': 'exasol-script-languages-container-tool',
2929
'version': '0.2.1',
3030
'description': 'Script Languages Container Tool',
31-
'long_description': "# Script-Languages-Container-Tool\n\n## Overview\n\nThe Script-Languages-Container-Tool (exaslct) is the build tool for the script language container.\nYou can build, export and upload script-language container from so-called flavors \nwhich are description how to build the script language container. You can find pre-defined flavors \nin the [script-languages-release](https://github.com/exasol/script-languages-release) repository. \nThere we also described how you could customize these flavors to your needs.\n\n## In a Nutshell\n\n### Prerequisites\n\n#### For installation\n\nIn order to install this tool, your system needs to provide \nthe following prerequisites:\n\n* Software\n * Linux\n * [bash](https://www.gnu.org/software/bash/) >= 4.2\n * [coreutils](https://www.gnu.org/software/coreutils/)\n * sha512sum\n * sed\n * [curl](https://curl.se/)\n\n#### For running\n\nIn order to use this tool, your system needs to fulfill the following prerequisites:\n\n* Software\n * Linux\n * [bash](https://www.gnu.org/software/bash/) >= 4.2\n * [coreutils](https://www.gnu.org/software/coreutils/)\n * readlink with -f option (the readlink version of macOS doesn't support -f)\n * realpath \n * dirname\n * [Docker](https://docs.docker.com/) >= 17.05 \n * with support for [multi-stage builds required](https://docs.docker.com/develop/develop-images/multistage-build/)\n * host volume mounts need to be allowed\n \n* System Setup \n * We recommend at least 50 GB free disk space on the partition \n where Docker stores its images, on linux Docker typically stores \n the images at /var/lib/docker.\n * For the partition where the output directory (default: ./.build_output)\n is located we recommend additionally at least 10 GB free disk space.\n\nFurther, prerequisites might be necessary for specific tasks. These are listed under the corresponding section.\n\n### Installation\n\nDownload the installation and update script via:\n\n```\ncurl -L -o install_or_update_exaslct.sh https://raw.githubusercontent.com/exasol/script-languages-container-tool/main/installer/install_or_update_exaslct.sh\n```\n\nBefore you continue with installation, please compute with the following command \nthe sha512 hash sum of the downloaded file and compare it with its \n[checksum file](installer/checksums/install_or_update_exaslct.sh.sha512sum):\n\n```\nsha512sum install_or_update_exaslct.sh\n```\n\nIf the checksums are identical, you can continue with the installation. \nPer default, the script installs exaslct into the current working directory.\nIt creates a script directory `exaslct_scripts` and the symlink `exaslct`\nto the starter script. If you want to change the path to the script directory \nyou can set the environment variable `EXASLCT_INSTALL_DIRECTORY` and \nif you want to create the symlink somewhere else you can set `EXASLCT_SYM_LINK_PATH`. \n\n```\nbash install_or_update_exaslct.sh [version|git-commit-id|branch|tag] \n```\n\nYou can use the same script to change the version of your current installation.\nYou only need to provide a different version, git-commit-id, branch or tag. \n\n### Usage\n\n#### How to build an existing flavor?\n\nCreate the language container and export it to the local file system\n\n```bash\n./exaslct export --flavor-path=flavors/<flavor-name> --export-path <export-path>\n```\n\nor upload it directly into the BucketFS (currently http only, https follows soon)\n\n```bash\n./exaslct upload --flavor-path=flavors/<flavor-name> --database-host <hostname-or-ip> --bucketfs-port <port> \\ \n --bucketfs-username w --bucketfs-password <password> --bucketfs-name <bucketfs-name> \\\n --bucket-name <bucket-name> --path-in-bucket <path/in/bucket>\n```\n\nOnce it is successfully uploaded, it will print the ALTER SESSION statement\nthat can be used to activate the script language container in the database.\n\n#### How to activate a script language container in the database\n\nIf you uploaded a container manually, you can generate the language activation statement with\n\n```bash\n./exaslct generate-language-activation --flavor-path=flavors/<flavor-name> --bucketfs-name <bucketfs-name> \\\n --bucket-name <bucket-name> --path-in-bucket <path/in/bucket> --container-name <container-name>\n```\n\nwhere \\<container-name> is the name of the uploaded archive without its file extension. To activate the language, execute the generated statement in your database session to activate the container for the current session or system wide.\n\nThis command will print a SQL statement to activate the language similar to the following one:\n\n```bash\nALTER SESSION SET SCRIPT_LANGUAGES='<LANGUAGE_ALIAS>=localzmq+protobuf:///<bucketfs-name>/<bucket-name>/<path-in-bucket>/<container-name>?lang=<language>#buckets/<bucketfs-name>/<bucket-name>/<path-in-bucket>/<container-name>/exaudf/exaudfclient[_py3]';\n```\n\n**Please, refer to the User Guide for more detailed information, how to use exalsct.**\n\n## Features\n\n* Build a script language container as docker images\n* Export a script language container as an archive which can be used for extending Exasol UDFs\n* Upload a script language container as an archive to the Exasol DB's BucketFS\n* Generating the activation command for a script language container\n* Can use Docker registries, such as Docker Hub, as a cache to avoid rebuilding image without changes\n* Can push Docker images to Docker registries\n* Run tests for you container against an Exasol DB (docker-db or external db)\n\n## Limitations\n\n* Caution with symbolic links: \n If you use symbolic links inside any directory of the command line arguments\n they must not point to files or directories outside the root of the path of the \n command line argument (i.e. --flavor-path ./flavors/my_flavor/ => There must be no symbolic\n link inside ./flavors/my_flavor point to anywhere outside of ./flavors/my_flavor).\n Background: Local directories paths must be mounted manually to the docker container. \n We currently support only the mounting of the given command line arguments, but we do not analyze\n the content of those directories.\n Plan is to fix this limitation with [#35](https://github.com/exasol/script-languages-container-tool/issues/35)\n \n## Table of Contents\n\n### Information for Users\n\n* [User Guide](doc/user_guide/user_guide.md)\n* [Changelog](doc/changes/changelog.md)\n\n## Information for Developers\n\n* [Developer Guide](doc/developer_guide/developer_guide.md)\n* [Dependencies](doc/dependencies.md)\n",
31+
'long_description': "# Script-Languages-Container-Tool\n\n## Overview\n\nThe Script-Languages-Container-Tool (exaslct) is the build tool for the script language container.\nYou can build, export and upload script-language container from so-called flavors \nwhich are description how to build the script language container. You can find pre-defined flavors \nin the [script-languages-release](https://github.com/exasol/script-languages-release) repository. \nThere we also described how you could customize these flavors to your needs.\n\n## In a Nutshell\n\n### Prerequisites\n\n#### For installation\n\nIn order to install this tool, your system needs to provide \nthe following prerequisites:\n\n* Software\n * Linux\n * [bash](https://www.gnu.org/software/bash/) >= 4.2\n * MacOsX\n * [bash](https://www.gnu.org/software/bash/) > 3.2\n * [coreutils](https://www.gnu.org/software/coreutils/)\n * sha512sum\n * sed\n * [curl](https://curl.se/)\n\n\n#### For running\n\nIn order to use this tool, your system needs to fulfill the following prerequisites:\n\n* Software\n * Linux\n * [bash](https://www.gnu.org/software/bash/) >= 4.2\n * [coreutils](https://www.gnu.org/software/coreutils/)\n * readlink with -f option\n * realpath \n * dirname\n * MacOsX (Please see limitations on [MacOsX](#macosx-limitations))\n * [bash](https://www.gnu.org/software/bash/) >= 3.2\n * [coreutils](https://www.gnu.org/software/coreutils/)\n * greadlink with -f option\n * realpath \n * dirname\n * [Docker](https://docs.docker.com/) >= 17.05 \n * with support for [multi-stage builds required](https://docs.docker.com/develop/develop-images/multistage-build/)\n * host volume mounts need to be allowed\n \n* System Setup \n * We recommend at least 50 GB free disk space on the partition \n where Docker stores its images, on linux Docker typically stores \n the images at /var/lib/docker.\n * For the partition where the output directory (default: ./.build_output)\n is located we recommend additionally at least 10 GB free disk space.\n\nFurther, prerequisites might be necessary for specific tasks. These are listed under the corresponding section.\n\n### Installation\n\nDownload the installation and update script via:\n\n```\ncurl -L -o install_or_update_exaslct.sh https://raw.githubusercontent.com/exasol/script-languages-container-tool/main/installer/install_or_update_exaslct.sh\n```\n\nBefore you continue with installation, please compute with the following command \nthe sha512 hash sum of the downloaded file and compare it with its \n[checksum file](installer/checksums/install_or_update_exaslct.sh.sha512sum):\n\n```\nsha512sum install_or_update_exaslct.sh\n```\n\nIf the checksums are identical, you can continue with the installation. \nPer default, the script installs exaslct into the current working directory.\nIt creates a script directory `exaslct_scripts` and the symlink `exaslct`\nto the starter script. If you want to change the path to the script directory \nyou can set the environment variable `EXASLCT_INSTALL_DIRECTORY` and \nif you want to create the symlink somewhere else you can set `EXASLCT_SYM_LINK_PATH`. \n\n```\nbash install_or_update_exaslct.sh [version|git-commit-id|branch|tag] \n```\n\nYou can use the same script to change the version of your current installation.\nYou only need to provide a different version, git-commit-id, branch or tag. \n\n### Usage\n\n#### How to build an existing flavor?\n\nCreate the language container and export it to the local file system\n\n```bash\n./exaslct export --flavor-path=flavors/<flavor-name> --export-path <export-path>\n```\n\nor upload it directly into the BucketFS (currently http only, https follows soon)\n\n```bash\n./exaslct upload --flavor-path=flavors/<flavor-name> --database-host <hostname-or-ip> --bucketfs-port <port> \\ \n --bucketfs-username w --bucketfs-password <password> --bucketfs-name <bucketfs-name> \\\n --bucket-name <bucket-name> --path-in-bucket <path/in/bucket>\n```\n\nOnce it is successfully uploaded, it will print the ALTER SESSION statement\nthat can be used to activate the script language container in the database.\n\n#### How to activate a script language container in the database\n\nIf you uploaded a container manually, you can generate the language activation statement with\n\n```bash\n./exaslct generate-language-activation --flavor-path=flavors/<flavor-name> --bucketfs-name <bucketfs-name> \\\n --bucket-name <bucket-name> --path-in-bucket <path/in/bucket> --container-name <container-name>\n```\n\nwhere \\<container-name> is the name of the uploaded archive without its file extension. To activate the language, execute the generated statement in your database session to activate the container for the current session or system wide.\n\nThis command will print a SQL statement to activate the language similar to the following one:\n\n```bash\nALTER SESSION SET SCRIPT_LANGUAGES='<LANGUAGE_ALIAS>=localzmq+protobuf:///<bucketfs-name>/<bucket-name>/<path-in-bucket>/<container-name>?lang=<language>#buckets/<bucketfs-name>/<bucket-name>/<path-in-bucket>/<container-name>/exaudf/exaudfclient[_py3]';\n```\n\n**Please, refer to the User Guide for more detailed information, how to use exalsct.**\n\n## Features\n\n* Build a script language container as docker images\n* Export a script language container as an archive which can be used for extending Exasol UDFs\n* Upload a script language container as an archive to the Exasol DB's BucketFS\n* Generating the activation command for a script language container\n* Can use Docker registries, such as Docker Hub, as a cache to avoid rebuilding image without changes\n* Can push Docker images to Docker registries\n* Run tests for you container against an Exasol DB (docker-db or external db)\n\n## Limitations\n\n* Caution with symbolic links: \n If you use symbolic links inside any directory of the command line arguments\n they must not point to files or directories outside the root of the path of the \n command line argument (i.e. --flavor-path ./flavors/my_flavor/ => There must be no symbolic\n link inside ./flavors/my_flavor point to anywhere outside of ./flavors/my_flavor).\n Background: Local directories paths must be mounted manually to the docker container. \n We currently support only the mounting of the given command line arguments, but we do not analyze\n the content of those directories.\n Plan is to fix this limitation with [#35](https://github.com/exasol/script-languages-container-tool/issues/35)\n\n### MacOsX Limitations\n \n* On MacOsX all arguments (flavors path, output directory, etc.) must point to locations within the current directory (background is that the MacOsX version does not support mount binding additional directories).\n\n## Table of Contents\n\n### Information for Users\n\n* [User Guide](doc/user_guide/user_guide.md)\n* [Changelog](doc/changes/changelog.md)\n\n## Information for Developers\n\n* [Developer Guide](doc/developer_guide/developer_guide.md)\n* [Dependencies](doc/dependencies.md)\n",
3232
'author': 'Torsten Kilias',
3333
'author_email': '[email protected]',
3434
'maintainer': None,
Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,28 @@
11
#!/usr/bin/env bash
22

3+
#####################################################################################
4+
###REMEMBER TO TEST ANY CHANGES HERE ON MACOSX!!!
5+
#####################################################################################
6+
37
set -euo pipefail
48

5-
SCRIPT_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
9+
rl=readlink
10+
if [[ "$(uname)" = Darwin ]]; then
11+
rl=greadlink
12+
fi
13+
14+
15+
if [[ ! "$(command -v $rl)" ]]; then
16+
echo readlink not available! Please install coreutils: On Linux \"apt-get install coreutils\" or similar. On MacOsX \"brew install coreutils\".
17+
exit 1
18+
fi
19+
20+
SCRIPT_DIR="$(dirname "$($rl -f "${BASH_SOURCE[0]}")")"
21+
622
PROJECT_ROOT_DIR="$SCRIPT_DIR/.."
723

824
IMAGE_NAME="$("$SCRIPT_DIR/construct_docker_runner_image_name.sh")"
925

1026
docker build -t "$IMAGE_NAME" -f "$SCRIPT_DIR/Dockerfile" "$PROJECT_ROOT_DIR" 1>&2
1127

12-
echo "$IMAGE_NAME"
28+
echo "$IMAGE_NAME"
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
06c592b5dc270c7b9f73a6343a98375b43f25273d013922d0e9b7d923caf02f89267222435597aef1e636e17cc3a6e40a2f3eaababda852f64781eb37a324621 build_docker_runner_image.sh
1+
f072bb99a00b63b80f5c2126c059b787873ecb154973ea48dd05e609005fbc32168667dbf3ae42b277a2d89d92d60df22ffc25d052c354ba8d435749c851781f build_docker_runner_image.sh
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
af67ec0600650d540254c062b28b73ddcec1171733812025cb4eec01ed77191ddbb5223236f8e9470df9032ac8539c4065b9541946001e03a7bd875fea2387cb exaslct_within_docker_container.sh
1+
26f584f6dc72b01fcf37e6666534580415c1bec73899f7fa2959acd6dc7def7b965a1649e580af7ab503d0a139c02dca1e23183332ac236e0ae0d0dd8029c53a exaslct_within_docker_container.sh
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
fbe0b3fafce7aa34eca0018d8cbfe0f61a57674065a3c270564e8334c94292af0faa6f3f628e278b7e0c4f79cf12deaeeeb114ae91e66065dfcdc712f1b9ba44 exaslct_within_docker_container_slim.sh
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
eb85b4ac9b0a2dcb174e0cf8049e777cac80a31771f10683f1f8dd63fb8b1fe8f1961887241cdc90876b51916208726e8fbe9c8b35f54a5bb3b8b099d922c7b1 exaslct_within_docker_container_with_container_build.sh
1+
0a7214054767a177bcae54208511dde8c7ca2c179d9b2997965bbe20e4153a4252ab86ce9a12ac6b914949bdadef5b1f95f6ddaab238ff072c183055e23c889d exaslct_within_docker_container_with_container_build.sh

0 commit comments

Comments
 (0)