Deploylib

#Deploylib

Deploylib is a scala tool for setting up and interacting with a cluster compute nodes, either privately owned cluster or an Amazon EC2 cluster. It is also integrates with Mesos, the cluster management platform, so it is capable of launching and managing Mesos masters, slaves, and Mesos applications. In particular, there is a framework for running and managing Java or Scala services (jars) and web services (wars via Jetty).

Setting up deploylib

Prerequisites

Install SCADS (see the "How to install and build SCADS" section of the wiki Home). The recommended installation is via SBT.

Environment Variables

In addition to the prerequisites, you must set the following environment variables:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_KEY_NAME
• AWS_KEY_PATH
• EC2_PRIVATE_KEY
• EC2_CERT

Using deploylib

Once you have set up deploylib, you should be able to use it by doing the following:

1. Type bin/mvnconsole (warning: this is misleadingly named mvnconsole because it used to use Maven (i.e. mvn), however it not longer uses MVN. Thus the name may change to console soon). This should drop you into a scala console. Remember, tab completion should work here.
2. Type :load deploylib/setup.scala. This should execute several import statements, such as logging, mesos, etc. The output should look as follows:

  scala> :load deploylib/setup.scala
  Loading deploylib/setup.scala...
  import net.lag.logging.Logger
  import deploylib._
  import deploylib.ec2._
  import deploylib.mesos._
  import deploylib.rcluster._
  import net.lag.configgy._
  toFile: (str: String)java.io.File
  debug: Unit
  updateScads: Unit
  stopAllInstances: Unit

1. To get a list of all EC2 instances you have running, type: EC2Instance.activeInstances. It should return something like the following:

scala> EC2Instance.activeInstances
INF [20110107-18:12:32.375] ec2: Updated EC2 instances state
res0: List[deploylib.ec2.EC2Instance] = List(<EC2Instance i-c9b64ba5>, <EC2Instance i-25ad5049>, <EC2Instance i-27ad504b>)

Using deploylib with Mesos

You can use deploy lib to start and manage a Mesos cluster running on EC2. To familiarize yourself with Mesos, check out the Mesos homepage.

Setup a Mesos Cluster

If you are working on the RAD Lab demo, these commands should only be run by one person, who is the "Cluster Owner" whose EC2 credentials are used to launch the cluster (and thus others can't access the cluster via EC2 tools or with deploylibs functions which rely on those EC2 credentials)

• MesosEC2.startMaster - start a Mesos master type:
• MesosEC2.addSlaves(<num_slaves_to_add>) - add a Mesos slave (which will spin up a new EC2 instance)
  • Note: You can call halt() on any EC2Instance
• MesosEC2.updateDeploylib - pushes the newest deploylib jars to the slaves from S3.