-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add script for setting up an AMI #11
base: master
Are you sure you want to change the base?
Changes from all commits
7a370ea
b0ded61
f924b5b
4c767d7
929ade1
e899b18
d5679c7
539d25c
7bf2b74
df3e9b8
354d18f
ddd0ddd
819d93c
1d9062c
9c2c6b9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
name: ramp | ||
channels: | ||
- conda-forge | ||
dependencies: | ||
- numpy | ||
- pyyaml | ||
- psycopg2 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this will not really work as desired I think since you defined There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for catching the mismatch. I switched from There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. conda already provides binary packages, so there should not exist such a thing (It's rather strange practice of the psycopg2 devs to do this I think) |
||
- sqlalchemy | ||
- boto3 | ||
- psutil | ||
- cloudpickle | ||
- gitpython | ||
- pip: | ||
- memory_profiler | ||
- https://api.github.com/repos/paris-saclay-cds/ramp-backend/zipball/master | ||
- https://api.github.com/repos/pythonprofilers/memory_profiler/zipball/master | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,123 @@ | ||
#!/bin/bash | ||
# | ||
# Setup a RAMP backend architecture for a given RAMP project | ||
|
||
set -e | ||
|
||
# Script variables definition | ||
project_name="$1" | ||
ramp_dependencies="$2" | ||
|
||
RAMPKIT_DIR="$HOME/ramp-kits" | ||
kit_url="https://github.com/ramp-kits/$project_name" | ||
kit_dir="$RAMPKIT_DIR/$project_name" | ||
|
||
ami_environment="$kit_dir/ami_environment.yml" | ||
data_dir="$kit_dir/data" | ||
|
||
# Text management | ||
bold=`tput bold` | ||
normal=`tput sgr0` | ||
underline=`tput smul` | ||
|
||
############## | ||
# Script help | ||
############## | ||
usage() | ||
{ | ||
echo " | ||
${bold}Setup a RAMP backend architecture for a given RAMP project${normal} | ||
|
||
Usage: | ||
|
||
${bold}./ubuntu_setup.sh <ramp_kit_name> <ramp_environment_file>${normal} | ||
|
||
where <ramp_kit_name> stands for the name of the RAMP kit on GitHub | ||
${underline}https://github.com/ramp-kits/<ramp_kit_name>${normal} | ||
|
||
where <ramp_environment_file> is the path to the `ramp_environment.yml` | ||
provided with this script | ||
" | ||
} | ||
|
||
####################################################### | ||
# Batch install of latest Miniconda in $HOME directory | ||
####################################################### | ||
miniconda_install() { | ||
LATEST_MINICONDA="http://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh" | ||
wget -q $LATEST_MINICONDA -O ~/miniconda.sh | ||
bash ~/miniconda.sh -b -p $HOME/miniconda3 | ||
|
||
echo 'export PATH="${HOME}/miniconda3/bin:$PATH"' >> ~/.profile | ||
source ~/.profile | ||
|
||
conda update --yes --quiet conda | ||
pip install --upgrade pip | ||
} | ||
|
||
################################################################# | ||
# Install Python dependencies in the base conda environment | ||
# | ||
# Argument: | ||
# path to a conda environment file (usually 'environment.yml') | ||
################################################################# | ||
update_conda_env() { | ||
environment_file=$1 | ||
conda env update --name base --file $environment_file | ||
} | ||
|
||
############################################################# | ||
# Upgrade the kernal and install conda and RAMP dependencies | ||
############################################################# | ||
system_setup() { | ||
sudo apt-get update | ||
sudo apt-get upgrade --yes | ||
miniconda_install | ||
update_conda_env $ramp_dependencies | ||
} | ||
|
||
################################################################# | ||
# Clone the ramp-kit, install dependencies and download the data | ||
################################################################# | ||
project_setup() { | ||
mkdir $RAMPKIT_DIR | ||
git clone $kit_url $kit_dir | ||
update_conda_env $ami_environment | ||
rm -rf $data_dir && mkdir $data_dir | ||
echo | ||
echo "${bold}==> Last step is to put the backend data into '$data_dir/'${normal}" | ||
echo | ||
} | ||
|
||
|
||
start_prompt() { | ||
echo | ||
echo "---------------------------------------------------------------" | ||
echo " Starting the setup of the RAMP AMI for $project_name project" | ||
echo "---------------------------------------------------------------" | ||
echo | ||
} | ||
|
||
end_prompt() { | ||
echo | ||
echo "---------------------------------------------------------------" | ||
echo " Ending the setup of the RAMP AMI for $project_name project" | ||
echo "---------------------------------------------------------------" | ||
echo | ||
} | ||
|
||
####### | ||
# Main | ||
####### | ||
main() { | ||
if [[ "$project_name" == "" || "$project_name" == "help" ]]; then | ||
usage | ||
else | ||
start_prompt | ||
system_setup | ||
project_setup | ||
end_prompt | ||
fi | ||
} | ||
|
||
main |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this file used for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still had in mind we should separate the "backend" requirements for the moment.
Since we install
ramp-backend
andramp-workflow
via pip, all of their dependencies are also installed via pip by default. If we specify their dependencies in this file, we then can use conda for most of these dependencies.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, missed it in the script that you do the
conda env update
twice, with this one and the one of the kit