AWS:EC2 - Change InstanceType to Scale Vertically - Add Support

[franklinwise]

Would be very helpful to be running a small instance, a database or some other persistent datastore and be able to scale that instance up (or down) vertically to a different instance type. (horizontally is supported very well by amzn).

So changing the instance_type from "m1.small" to "m1.large" would launch a new ec2 instance and then reattach the "old" EBS volumes to the new instance, and ideally migrate the IP address as well before deleting the old instance.

Basically enabling one to grow a small replicated persisted cluster vertically until it makes sense to start scaling horizontally.

resource "aws_instance" "web" {
    ami = "ami-1234"
    instance_type = "m1.small"
    ebs_block_device {
         volume_size = "100gb"
    }
    tags {
        Name = "couchbase-01"
    }
}

resource "aws_instance" "web" {
    ami = "ami-1234"
    instance_type = "m1.large"
    ebs_block_device {
         volume_size = "100gb"
    }
    tags {
        Name = "couchbase-01"
    }
}

[radeksimko]

👍 I like the idea. I've been thinking about similar functionality for RDS, where I can see this being even more important and useful, especially when upgrading your set of DB slaves for any reason.

Even though I can't think of anyone explicitly wanting to do it the risky way (destroying & creating a new instance), it's worth mentioning that the cost for the redundancy and safe upgrade is the cost for running two instances in parallel at the same time.

[franklinwise]

Turns out amazon allows one to change the instance type without destroying the machine and it will correctly launch the same instance on a larger vm providing the underlying hardware virtualization type is the same. So in this case, there is much less for terraform to do on AWS. From what I understand, the process is stop the instance, change the instance type, start the instance and viola! Thus, instead of a change of instance type causing terraform to mark the instance for deletion, it would be marked for change and reboot (or something like that).

[neurogenesis]

👍 (for @franklinwise 's reply). There's no need to re-provision an instance to change instance type (so long as the instance is EBS backed and the new type is permitted, some types are available only in VPCs and others are prohibited in certain availability zones). This is also true of the EBS Optimized option.

This seems like a lot of provider-specific logic to add though, vs. making the change via AWS UI/CLI and updating the terraform state file (for something that should be happening rarely).

On 0.4.2 and 0.5.3, changing the value in the terraform config before manually modifying the instance will result in removing and re-creating the instance. To avoid this you can manually change the instance first, but you have to update the state file.

The problem is that terraform refresh doesn't update the state file after you modify the instance, so you have to directly edit the state file:

$ terraform refresh
...
aws_instance.ec2server: Refreshing state... (ID: i-xxxxxxxx)
...

$ terraform plan
...
-/+ aws_instance.ec2server
    ami:                                           "ami-83c525e8" => "ami-83c525e8"
    availability_zone:                             "us-east-1b" => "<computed>"
    ebs_block_device.#:                            "1" => "<computed>"
    ebs_optimized:                                 "true" => "1"
    ephemeral_block_device.#:                      "1" => "1"
    ephemeral_block_device.430514000.device_name:  "/dev/sdb" => "/dev/sdb"
    ephemeral_block_device.430514000.virtual_name: "ephemeral0" => "ephemeral0"
    iam_instance_profile:                          "ec2-bootstrap-cloudinit" => "ec2-bootstrap-cloudinit"
    instance_type:                                 "r3.xlarge" => "r3.2xlarge" (forces new resource)
...

(related to #1571, but also still true for 0.5.3)

[CpuID]

@neurogenesis I believe your issue is fixed in #2365 - just tested on 0.6.0 (I initially thought it was broken, but turns out my .tf file had the wrong instance_type still so terraform was trying to do it's job).

To make this happen, should just be a StopInstances + ModifyInstanceAttribute + StartInstances call (in that order).

[saulshanabrook]

So does terraform now update an instance instead of creating and destroying it when the instance type is changed?

[saulshanabrook]

And it will re-attach EBS devices?

[xeizmendi]

I have just tested this with terraform v0.6.6 and it destroys the instance instead of stopping, changing the instance type and starting :(

[theist]

AWS only lets you to change instance type if you stop the instance, so "scaling vertically" is done in three steps: stop instance, change its type (modify instance), start instance. In an VPC it will retain its public and private addresses, but not in classic accounts. This is beyond terraform capabilities because terraform (afaik, i'm only a user) does not know nothing about machine states

[catsby]

Hey all – this kind of multi-step process is a thing we would like to add, but in all honesty would require some significant work done on the core of Terraform first, and is a thing that is pretty far off. In light of that I'm going to close this issue for now. Please let me know if you have other questions.

[salewski]

For folks landing here from a google search: it looks like this feature was implemented in issue #9157 (see pull #11998), which was included in the terraform-0.8.8 release (2017-03-02).

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.