UPDATE: We have no plans to continue updating SGT moving forward. Please feel free to clone/fork this project, or try Kolide Fleet as an alternative (https://www.kolide.com/fleet/).

SGT: OSQuery Management Server Built Entirely on AWS!

SGT is an osquery management server written in Golang and built in aws. Sgt (Simple Go TLS) is backed entirely by AWS services, making its infrastructure requirements extremely simple, robust and scalable.

SGT is managed entirely through terraform

NOTE: SGT is under active development. Please help us improve by submitting issues!

Getting started.

NOTE If you are upgrading from a previous version, please see the release notes for 0.2.0

Getting started with sgt is designed to be very simple with minimal setup required. To get started, however, you will need a FEW things first.

⚠️ There is current an issue with terraform version 11.5-11.10 that will cause terraform to error out during destroy. The current workaround is to set environment variable TF_WARN_OUTPUT_ERRORS=1 ⚠️

prereqs:

1. An AWS account with admin access to DynamoDB, EC2, ES (ElastisearchService), Kinesis/Firehose and IAM. (note, this must be programatic access, so you can have an access key and secret to use)
2. Golang 1.9.0+
3. Terraform 11.9+
4. A domain with DNS managed via Route53 (Note: This does not mean you need to buy a domain, you can use an existing domain and just manage DNS on Route53)
5. An SSL cert with public and private keypair. This will be used to terminate TLS connections to our server see Obtaining a free ssl cert for SGT with Letsencrypt for one method of aquiring a certificate
6. An aws profile configured.

Installation

1. Clone the repo

   git clone [email protected]:OktaSecurityLabs/sgt.git $GOPATH/src/github.com/oktasecuritylabs/sgt
   

2. change into the downloaded directory

   cd $GOPATH/src/github.com/oktasecuritylabs/sgt
   

3. Build the project

   go build
   

4. Copy your ssl certs to the proper directory. For this example, I'm using a subdomain of example.com with a letsencrypt certificate, sgt-demo.example.com. Lets encrypt certs live in /etc/letsencrypt/live/<site> so I'm copying them from there into the cert directory for SGT.

   sudo cp /etc/letsencrypt/live/sgt-demo.example.com/fullchain.pem certs/fullchain.pem
   sudo cp /etc/letsencrypt/live/sgt-demo.example.com/privkey.pem certs/privkey.pem
   

5. Rename your certs to reflect which site they belong to. I recommend following the example format of

   example.domain.com.fullchain.pem
   

   moving...

    cd certs
    mv fullchain.pem sgt-demo.example.com.fullchain.pem
    mv privkey.pem sgt-demo.example.com.privkey.pem
    cd ..
   

6. Create a new environment by following the prompts

   ./sgt wizard
   

   6a. Enter a name for your environment (I'm calling my demo one sgt-demo)

   Enter new environment name.  This is typically something like'Dev' or 'Prod' or 'Testing, but can be anything you want it to be: sgt-demo
   

   6b. Choose the AWS profile to use (Mine is again called sgt-demo)

   Enter the name for the aws profile you'd like to use to deploy this environment
   if you've never created a profile before, you can read more about how to do this here
   http://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html
   a 'default' profile is created if you've installed and configured the aws cli:
   sgt-demo
   

   6c. Enter the IP address that you are currently deploying from.

   Enter an ipaddress or cidr block for access to your elasticsearch cluster.
   Note:  This should probably be your current IP address, as you will need to be able to access
   elasticsearch via API to create the proper indices and mappings when deploying: xxx.xxx.xxx.xxx/24
   

   6d. Name your log bucket. I recommend something easily identified for your domain.

   Enter a name for the s3 bucket that will hold your osquery logs.
   Remeber, S3 bucket names must be globally unique: sgt-demo.log.bucket
   

   6e. And your config bucket...

   Enter a name for the s3 bucket that will hold your server configuration
   Remember, S3 bucket names must be globally unique:
   sgt-demo.configuration.bucket
   

   6f. Enter your root domain

   Enter the domain you will be using for your SGT server.
   Note:  This MUST be a domain which you have previously registered or are managing throughaws.
   This will be used to create a subdomain for the SGT TLS endpoint
   example.com
   

   6g. Enter the subdomain (sgt-demo in my case)

   Enter a subdomain to use as the endpoint.  This will be prepended to the
   domain you provided as a subdomain
   sgt-demo
   

   6h. Enter your aws keypair name

   Enter the name of your aws keypair.  This is used to access ec2 instances ifthe need
   should ever arise (it shouldn't).
   NOTE:  This is the name of the keypair EXCLUDING the .pem flie name and it must already exist in aws
   my-secret-key-name
   

   6i. Enter the name of your keypair and priv key, as you named them above.

   Enter the name of the full ssl certificate chain bundle you will be using for
   your SGT server.  EG - full_chain.pem :
   sgt-demo.example.com.fullchain.pem
   Enter the name of the private key for your ssl certificate.  Eg - privkey.pem:
   sgt-demo.example.com.privkey.pem
   

   6j. Enter the node secret

   Enter the node secret you will use to enroll your endpoints with the SGT server
   This secret will be used by each endpoint to authenticate to your server:
   my-super-secret-node-secret
   

   6k. Enter the app secret

   Enter the app secret key which will be used to generate session tokens when
   interacting with the API as an authenticated end-user.  Make this long, random and complex:
   diu3piqeujr302348u33rqwu934r1@#)(*@3
   

   Select N when prompted to continue. Because this is a demo environment, we're going to make a small change to our configuration.

7. Edit the environment config file found in /terraform/<environment/environment.json with your favorite editor and change the value for create_elasticsearch to 0. This will disable the creation of elasticsearch, which we will not be using for this demo. In a production environment, Elasticsearch would be a large part of your process, but it adds significant cost and it's not needed for this demo.

   {
     "environment": "example_environment",
     "aws_profile": "default",
     "user_ip_address": "127.0.0.1",
     "sgt_osquery_results_bucket_name": "example_log_bucket_name",
     "sgt_config_bucket_name": "example_config_bucket_name",
     "domain": "somedomain.com",
     "subdomain": "mysubdomain",
     "aws_keypair": "my_aws_ec2_keypair_name",
     "full_ssl_certchain": "full_cert_chain.pem",
     "ssl_private_key": "privkey.pem",
     "sgt_node_secret": "super_sekret_node_enrollment_key",
     "sgt_app_secret": "ultra_mega_sekret_key_you'll_never_give_to_anyone_not_even_your_mother",
     "create_elasticsearch": 0
   }

Deploy!!

Its finally time to deploy, although hopefully that wasn't too painful. Deployment is by far the easiest part.

./sgt deploy -env <your environment name> -all

This will stand up the entire environment, including endpoint configuration scripts which we will use to set up some osquery nodes later. The entire process should take about 5-10 minutes depending on your internet connection, at which point you should be ready to install osquery on an endpoint and start receiving logs!

-Note: This getting started guide originally appeared on blog.securelyinsecure.com, but I'm appropriating it for the docs as well, due to it being better than the last readme I wrote.

Once you've installed Go and Terraform, and built your SGT binary, its time to run your deployment!

The wizard will walk you through everything you need to configure a new environment, create the proper directory structure and the environment specific configuration files and stand up the environment if you choose to do so

./sgt wizard

Among other things, the wizard will ask you to provide: The "mail domain for the users of your Kibana dashboard". This should be the domain name used for the email addresses of the people who will be using the Kibana dashboard (example: company.com)

A comma delimited list of users for the Kibana dashboard. The users in the list must correspond to email addresses of the users. For example, if you wanted to initialize Kibana with 2 users (Some Guy, [email protected]; Someone Else, selse@company,com) your input at this prompt woukld be sguy,selse

When you are done with the wizard, you will be prompted to either continue to deploy the actual resources, or exit. If you choose to exit, you you will need manually deploy later

Manual deployment

SGT can be deployed as a full environment, or individual pieces(Note that the components still requires their dependencies to be built, they may just be updated individually to save time)

To deploy SGT...

./sgt deploy -env <environment> -all

To deploy/update an individual component..

./sgt deploy -env <environment> -components elasticsearch,firehose

for a full list of commands, issue the -h flag

If terraform fails at any point during this process, cancel the installation ctrl+c and review your errors. SGT depends on all previous deploy steps completing successfully, so it is important to make sure this occurs before moving on to next steps

Creating your first user.

To create a user to interact with SGT, pass the -create_user flag with the requisite options

./sgt create-user -credentials-file <cred_file> -profile <profile> -username <username> -role <"Admin"|"User"|"Read-only">

Getting an Authentication Token

Using any portion of the End-user facing API requires an Authorization token. To get an auth token, send a post request to /api/v1/get-token supplying your username and password in the post body

{"username": "my_username", "password": "my_password"}

If your credentials are valid, you will recieve a json response back

{"Authorization": "<long jtw">}

Provide this token in any subsequent requests in the Authorization header

Creating Additional Kibana Users Post-Deployment

1. Log into the AWS account where you deployed sgt, and go to the cognito service page
2. Click Manage User Pools
3. Click the User Pool you created during the sgt deployment
4. Click Users and Groups
5. Click Create User
6. In the Username text box, type the username portion of the user's email address
7. Leave the box "Send an invitation to this new user?" checked
8. Check the "Email" box
9. Un-check the "Mark phone number as verified?" box
10. In the Email text box, type the user's email address
11. Click Create User

Documentation notes:

Documentation is lacking right now due to a rather un-fun flu season. However, updates to documentation should be expected in teh coming week or so. (This note marked: 1/17/18)