Skip to content
This repository has been archived by the owner on May 12, 2021. It is now read-only.

Feature add docker/volume and volume/secret support (#58) #59

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

JustinVenus
Copy link

@JustinVenus JustinVenus commented Jun 21, 2019

Description:

Implement `docker/volume` and `volume/secret` support on the scheduler and python client.

* update docs for using docker/volume and volume/secret

* implement scheduler changes for docker/volume and volume/secret

* implement python thrift changes for docker/volume and volume secret

* update python thrift tests for docker/volume and volume/secret

* update aurora client schema for docker/volume and secret/volume

* create/update structs for docker/volume and volume/secret isolators

Testing Done:

This patch was originally applied to rel/0.21.0 and has been updated to apply to the current master branch. In the form applied to rel/0.21.0 it has been running in my production environments since December 2018.

As far as how to test the new features.

docker/volume

I have dvdcli-0.2.0. I run rexray for EBS support. The mesos-agent uses AWS IAM to describe/attach/detach/etc EBS volumes.

# /etc/rexray/config.yml
libstorage:
  service: ebs
  integration:
    volume:
      operations:
        mount:
          preempt: true
        remove:
          force: true
        create:
          default:
            size: 2
            type: gp2
            fsType: xfs
  server:
    services:
      ebs:
        driver: ebs
ebs:
  maxRetries:         10
  statusMaxAttempts:  10
  statusInitialDelay: 100ms
  statusTimeout:      60s
# /etc/systemd/system/multi-user.target.wants/rexray.service
[Unit]
Description=REX-Ray Storage Orchestration Engine
After=network-online.target cloud-config.target cloud-init-local.service

[Service]
Environment=DOCKER_LEGACY=true
Restart=always
ExecStart=/usr/bin/rexray service start -f

[Install]
WantedBy=multi-user.target
volume/secret

This is a little more interesting to test. I'm not aware of any publicly available secret resolver implementations. At minimum you have to write your own secret resolver. We chose to target with hashicorp/vault and implent a kv2 resolver.

    * update docs for using docker/volume and volume/secret

    * implement scheduler changes for docker/volume and volume/secret

    * implement python thrift changes for docker/volume and volume secret

    * update python thrift tests for docker/volume and volume/secret

    * update aurora client schema for docker/volume and secret/volume

    * create/update structs for docker/volume and volume/secret isolators
builder.setContainer(Container.mesos(
container.newBuilder()
.setVolumes(container.getVolumes().stream()
.map(v -> v.isSetVolumeType() ? v.newBuilder() : v.newBuilder()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since in the api.thrift we set a default value on line 223 https://github.com/apache/aurora/pull/59/files#diff-03d58f7b857e5429e1668264857798e1R223 do we ever hit the case where the volume is not set? Or does thrift not assign anything if we don't explicitly set anything? Just wondering because having a value be optional and have a default value at the same time is counter intuitive.

}

union VolumeSource {
1: string hostPath
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think by using this as a union, we can get away without using a VolumeType therefore shrinking our Thrift footprint and we reduce the error proneness of having to set the same value twice . (e.g. VolumeType== "X", VolumeSource == "X.value")

In the code we can check isSet and use that as the "type" to choose the code path. Since Thrift won't allow more than one field in the union to be set we can safely depend on this. We use this same pattern in the SLA strategy f2acf53#diff-dc70a52021b0286021f89682c3b0841bR241

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants