Deploying diego to bosh-lite fails on network capacity

[shalako]

"Error 430001: Could not reserve network for package compilation: capacity"

Output of bosh task 6 --debug: https://gist.github.com/shalako/8ab6016f97aa1dc71da7

Per the instructions on the release README, I have deployed cf-release first: https://gist.github.com/shalako/e244ba78b42331b8d028

@atulkc told me he worked around this by reducing the number of the compilation job instances in the manifest. So one fix could be to modify the bosh-lite stub to enable a more painless deployment experience.

compilation:
  cloud_properties: {}
  network: diego1
  reuse_compilation_vms: true
  workers: 4

But this seems like potentially a bosh-lite CPI thing, for it should support deploying cf-release and diego-release together. Opened issue for BOSH.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/97912668.

[shalako]

Verified that reducing workers from 6 to 4 results in successful deploy.

[emalm]

Sounds like the bosh-lite templates don't make enough dynamic IP addresses available in their network settings to accommodate all the compilation VMs. In particular, our infrastructure stubs allocate only 10 dynamic IPs in the diego1 subnet, and a default bosh-lite deployment uses 5 of them for VMs now, so that 6th compilation vm will fail.

I think we should change our bosh-lite stubs to match the conventions in cf-release's warden infrastructure stubs, where each named network covers a 10.244.x.0/24 of IPs, with 10.244.x.0/25 allocating a total of 32 static IPs and 10.244.x.128/25 allocating 32 dynamic IPs. Will prioritize the tracker story and add this context to it.

[emalm]

The updated network sections are now on diego-release's master branch. Can you confirm that they resolve this issue? Because of the changes in IP organization, you may need to delete your deployment or scale down certain jobs to avoid deployment errors on IP allocation conflicts.

[emalm]

Closing, as I think this is resolved now.