Skip to content

Setup

Jump to bottom Edit New page
Brad Smith edited this page Mar 25, 2017 · 58 revisions

Overview

There are four ways in which the provisioning system might be used:

  • Deploying to a vagrant-based VM
    • Best for development and testing, unless you need a non-x86 architecture
  • Deploying to a cubietruck
    • x2go default for deployments to locations with no existing hardware, or power/space limitations
  • Deploying to a commodity PC
    • Tunapanda default for deployments to locations that already have hardware
  • Refreshing/reloading the config after deployment
    • Applies to all of the above methods
    • bootstrap.sh is run from /etc/rc.local at every boot (but skips most tasks if not net connection is detected)

If something goes wrong...

It may seem pessimistic to start the setup instructions with this, but this software is in active development by a very small group of people, and though everything is tested prior to release, bugs may still get through, so here are some tips:

  • Please report bugs! File them as github issues, and/or ask questions in our discussion group.
  • The scripts/diag.sh tool produces an archive with useful information about the system. Please consider running this and attaching the results to bug reports.
  • If the provisioning process didn't have an obvious failure, but some services don't seem to be available, try rebooting the cubietruck, but, if you do this, note that install may have started a set of background rsync processes to download content files, and if you reboot they can die. When the machine comes back up,re-start them by running /usr/local/tunapanda/data/cron_datasync/syncscripts.d/* as root.

Deployment Options

Deploying to a vagrant VM:

  1. Install vagrant and virtualbox
  2. git clone https://github.com/tunapanda/provision
  3. cd provision
  4. See the Customization section below for ways you can fine-tune your install beyond the default setup.
  5. vagrant up
  6. To access the system, either:
  • vagrant ssh (CLI only, no password required)
  • A new VirtualBox window should appear during deployment. You can log in there with username teacher and password Tunapanda2. If you installed the GUI (see 'customization' below), you will have full desktop access here.
  1. See the Post-deplyment steps, below.

An important note about vagrant deployments: Normally, the deployment process clones the provision git repo into /usr/local/tunapanda/provision. In this case, you've already cloned the repo on your host system, so we can just use that. The Vagrantfile that controls the VM's behavior mounts the directory where you cloned the repo on your host as /usr/local/tunapanda/provision on the VM. This is why VM-based deployments can be very convenient for development. Just modify stuff in the repo and run vagrant provision to apply your changes.

Deploying to a cubietruck

edX requires several modifications to work on ARM. Thanks mainly to the work of Arturo Samanez and Ahmad Draidi, we have a prebuilt image of Ubuntu 14 for ARM (based on Igor Pecovnik's Cubietruck Ubuntu), with edX pre-installed. You will need to begin by using it to create an SD card from which to boot the cubie.

  1. (Optional) Normally, after the initial installation, several rsync processes will be spawned to start downloading site content in the background. You can also pre-download this content if it's easier for you to just do a few downloads overnight before you you begin. Just download these archives and put them on a USB drive or something else you can use to extract them onto the cubietruck later:
  1. Download the x2go initial setup image here.
  2. Write the image to your SD card
  • bzcat x2go_edx.img.bz2 | sudo dd bs=512 of=/dev/sdX
    • Replace X with your device's letter. Run dmesg after inserting the card for hints if you're not sure (Your device might also be located on /dev/mmcblkX).
  1. Plug your cubietruck's wired ethernet adapter into the network so it can access the Internet for installations later.
  2. Boot your cubietruck from the SD card.
  3. The initial boot may take a while. It should configure its self to use either VGA or HDMI, whichever is plugged in during the initial boot, but you don't even need video because...
  4. When the cubie finishes booting, you should see a wireless access point called x2go.
  5. Connect to the x2go wifi network and ssh to [email protected] with the password x2go
  6. Change the root password!
  • passwd
  1. (Optional) Use your cubietruck's SATA drive, if it has one. You have two options:
  • Store everything possible on the SATA drive. We're still testing, but it looks like this will improve performance, so we recommend it. Note that this moves most of the OS and all of the site data, but the cubie will still need the SD card to boot!
    1. ~/nand-sata-install
    • There is also currently a nand-sata-install.sh file. You can ignore it.
    1. Follow the instructions to install to /dev/sda1 Do not choose the default of /dev/nand2! It won't have enough space.
    • If this fails with a message about the disk not being partitioned, try this and then re-run nand-sata-install: dd if=/dev/zero of=/dev/sda bs=1M count=1 && parted /dev/sda 'mklabel msdos mkpart primary 1 -1' && partprobe /dev/sda
    1. Leave the SD card in the cubietruck (the cubie still needs it to boot, even with most of the OS on the SATA drive), reboot, and log in again.
  • Store just the site data (course content, etc) on the SATA drive. This lets you re-OS and reprovision from scratch if you need to without having to re-download content that probably won't change between provisionings.
    1. Be sure you are OK losing all data currently on the drive!
    2. sudo mkfs.ext4 -L TUNAPANDA_DATA /dev/sda1
    • The provisioning script will cause any device with the label TUNAPANDA_DATA to be auto-mounted under /usr/local/tunapanda/data/.
  1. (Optional) Set up a swapfile for virtual memory. This is highly recommended if you have a SATA drive, but you should probably not do it if you only have an SD card. It will use up 4G of disk space, but can greatly improve the performance of big services like edX.
  2. /usr/local/tunapanda/provision/scripts/create_swapfile.sh
  3. (Optional) Expand the previously-downloaded content (if you downloaded it in the first step).
  4. mkdir -p /usr/local/tunapanda/data/portal_modules
  5. cd /usr/local/tunapanda/data/portal_modules
  6. Expand the archives into this directory
  7. Update the provisioning scripts
  8. cd /usr/local/tunapanda/provision
  9. git pull Important! Don't skip this!
  10. (optional, unless you have limited storage space) Customize your deployment
  11. cd /usr/local/tunapanda/provision
  12. cp localconfig.yml.defaults localconfig.yml
  13. Edit localconfig.yml. If in doubt, don't change anything, but here are a couple of things you may want to change. For details, see the comments in the file. * Set the provision__instance, provision__site, and provision__domain vars to customize the hostname. * If you have limited storage space (e.g. no SATA drive), you should also include wikipedia__enabled: false and/or kalite__enabled: false, since kalite and wikipedia require more space than other services.
  14. Start the main provisioning
  15. Run byobu-enable. Network changes and service restarts during the provisioning process may cause your ssh connection to drop. byobu (similar to Gnu screen) will allow the process to continue running, and let you re-connect to it later. * If this does happen, try again periodically, noting that you may need to reconnect to the wireless network, too. * If it looks like it's not coming back, try plugging in a keyboard and mouse. * If all else fails, try rebooting and re-running the command in the next step.
    • ...and don't forget the If something goes wrong... section above!
  16. /usr/local/tunapanda/provision/scripts/bootstrap.sh
  17. Go to post-deployment steps.

Deploying to a commodity PC

Note: These instructions assume a machine already running Ubuntu 14 (other Debian-like distros will probably also work, but have not been tested).

Really important note: Following these instructions will completely reconfigure the system. Don't do this on a personal machine you don't plan to use as a dedicated server. If you're just curious, use the instructions for deploying on a virtual machine instead!

  1. cd /tmp
  2. wget https://github.com/tunapanda/provision/blob/master/scripts/bootstrap.sh
  3. sudo bash bootstrap.sh # add -x if troubleshooting

At this point some basic configuration been done, but mostly we've just set up the /usr/local/tunapanda/provisioning directory and created some users. To customize your deployment further, see Customizing below.

Post-deployment steps

If you installed the portal module (included by default in most setups)

In order for the content modules to be searchable, the database for the web crawler sphider must be populated. You can do this by running the following as root:

  1. wget http://geekdome.net/x2go/cubietruck_trusty_edx/sphider_plus.sql.bz2 * ...unless you pre-downloaded this using the link in the Deploying to a Cubietruck instructions!
  2. bzcat sphider_plus.sql.bz2 | mysql sphider_plus

Customizing

First, a very brief overview of Ansible, the system we use to configure systems during deployment. You will need to understand at least three concepts: roles, playbooks, and variables.

  • A role describes how to deploy a specific service or machine type. Most of the roles we use can be found in playbooks/roles/.
  • A playbook describes which roles to load, among other things. The file matched by playbooks/*.yml are all playbooks. By default, when you run scripts/bootstrap.sh, it uses playbooks/main.yml for deployment instructions.
  • A variable is a setting that can control the behavior of roles and playbooks.

Simple customization

If all you need to do is customize which services are installed, how many users are created, the system's hostname, etc, the easiest way to do this is with a localconfig.yml.

  1. cd /usr/local/tunapanda/provision
  2. cp localconfig.yml-sample localconfig.yml
  3. Open localconfig.yml in a text editor and make whatever customizations you need. The comments in the file should explain everything, but basically there are two things you can define here:
  • The playbook: setting lets you specify one of the playbooks in the playbooks/ directory, or you can leave it set to default, and a base system will be provisioned, plus any role enabled in the vars section further down.
  • If you specify an existing playbook, it will perform actions defined in playbooks/$PLAYBOOK.yml, modified by variables read from the following sources, from highest to lowest precedence:
    1. The vars section of your localconfig.yml
    2. playbooks/group_vars/$PLAYBOOK (if it exists)
    3. playbooks/group_vars/all
  • If you're familiar with Ansible, you may also note that it's possible to set vars in the playbook its self, but this doesn't affect things like playbooks loaded with include statments, which many of our playbooks use, so these are typically avoided.
  • If you are deploying on a cubietruck (or really anything with an ARM architecture), the provisioning system will automatically use the cubietruck playbook by default, and if you are deploying with Vagrant it will use the vm playbook by default.
  • Read the comments in localconfig.yml.sample for the rest!

You can also create a localconfig.yml with all available variables set to their default values like this:

(echo "---" ; echo "groups: [ default ]"; echo "vars:" ; for r in $(ls playbooks/roles/) ; do echo "" ; echo "  ## $r role" ; cat playbooks/roles/$r/defaults/main.yml | grep -v '^---' | grep -v '^[[:space:]]*$' | sed 's,^,  ,'; done) > localconfig.yml

To apply your changes, run sudo scripts/bootstrap.sh

Advanced customization

If the file playbooks/custom.yml exists, bootstrap.sh will use it instead of the default. If you know what you're doing with Ansible, you can use this to create your own custom playbook. Normally, the best profile is automatically chosen based on hardware architecture, etc, but if you want to (and you know what you're doing!), you can fine-tune things with a special profile called custom, and the localconfig.yml file. Note that this file does not exist by default, but you can create one using localconfig.yml.sample as a template, or using the following command, which will create a config file with all available settings set to their default values:

To apply your changes, run sudo scripts/bootstrap.sh

Creating an ISO

Note: this feature hasn't been tested much recently. If you encounter problems, please file an issue with information on how you used it and what went wrong

You can generate a bootable DVD of the provisioned system, which can then be used to demo or install a duplicate of the system elsewhere, by adding build_iso__enabled: true to your localconfig.yml.

Add a custom footer
Add a custom sidebar
Clone this wiki locally