Allow loading of sparkle packs

[patrobinson]

SparkleFormation supports the concept of a SparklePack which can be packaged as a RubyGem and required inside the template that utilises it. This allows re-usable dynamics to be shared very simply.

Unfortunately the implementation is fundamentally broken. Such that requiring a sparkle pack inside a template like so (which is what the documentation suggests):

require 'vpc_sparkle_pack'
pack = ::SparkleFormation::SparklePack.new(:name => 'vpc_sparkle_pack')
SparkleFormation.new(:util_vpc, sparkle: pack) do
...
end

results in a race condition, dependent on the file load order which is not deterministic.

This bug occurs when SparkleFormation tries to load a SparklePack while it's still loading templates/. Therefore the easiest workaround is to pre-load the SparklePack from StackMaster.

I would've liked to have fixed this in SparkleFormation, but after spending 3 days on the problem I feel I am no closer to a solution.

[gstib]

Is this meant to be sparkle_packs

[patrobinson]

Fixed in b593862

[gstib]

Might be worth stating that we are deviating from SparkleFormation prescribed technique.
i.e. You wont be doing this

require 'my_sparkle_pack'

custom_pack = SparkleFormation::SparklePack.new(:name => 'my_sparkle_pack')
sfn = SparkleFormation.new(:my_template) do
   dynamic!(:my_dynamic)
end
sfn.sparkle.add_sparkle(custom_pack)

It's just

SparkleFormation.new(:my_template) do
   dynamic!(:my_dynamic)
end

[patrobinson]

Added in 978dfea

[gstib]

This line is a little confusing as well. I can see it's gemname: packname but aren't they the same thing...
Can you get rid of the packname

[patrobinson]

They can be different, when you call register! it's possible to supply a different name.

[gstib]

The question is what value would it add to be different. Why would someone want/need to.

[patrobinson]

I don't know, but SparkleFormation supports it so I'm inclined to maintain that interface unless there's some compelling reason not to.

[gstib]

When you traditionally loaded like this

require 'my_sparkle_pack'

custom_pack = SparkleFormation::SparklePack.new(:name => 'my_sparkle_pack')
sfn = SparkleFormation.new(:my_template) do
   dynamic!(:my_dynamic)
end
sfn.sparkle.add_sparkle(custom_pack)

They actually needed to be the same to work

[patrobinson]

From what I understand it's entirely possible to load it like this:

require 'my_sparkle_pack'

custom_pack = SparkleFormation::SparklePack.new(:name => 'wow_dude_a_sparkle_pack')

Where my_sparkle_pack registers itself like so:

SparkleFormation::SparklePack.register!('wow_dude_a_sparkle_pack')

[gstib]

@patrobinson that's undocumented, you have to physically look at the code to see that. Why would you make it hard to consume by naming the pack different to the gem? Seems an odd thing to do to me.

If you look at the how they tell you to search for sparkle-pack gems here and look at gems from here

No-one that has implemented a pack has done that. They've all followed the doco from here

https://github.com/reverseskate/sparkle-pack-cfn-init-chef/blob/master/lib/sparkle-pack-cfn-init-chef.rb
https://github.com/timurb/sparkle-pack-shell-helpers/blob/master/lib/sparkle-pack-shell-helpers.rb
https://github.com/reverseskate/sparkle-pack-aws-availability-zones/blob/master/lib/sparkle-pack-aws-availability-zones.rb
https://github.com/reverseskate/sparkle-pack-aws-amis/blob/develop/lib/sparkle-pack-aws-amis.rb
https://github.com/carnivore-rb/sparkle-pack-jackal-cfn/blob/master/lib/sparkle-pack-jackal-cfn.rb
https://github.com/timurb/sparkle-pack-rails-helpers/blob/master/lib/sparkle-pack-rails-helpers.rb
https://github.com/reverseskate/sparkle-pack-cfn-init-chef/blob/master/lib/sparkle-pack-cfn-init-chef.rb

I wouldn't cater for an undocumented edge case personally. I'd make it easy to use and understand.

[patrobinson]

It's a supported API but whatever, done in 00e8993

[gstib]

You might need to change sparkle_packs to a list of sparkle packs to preserve order. I'm not sure if this structure will guarantee it. Ordering looks important if you want to override things. See here

[simpsora]

LGTM (I think)

[shivaman]

If Sparkleformation fixes it at some time in the future, will we revert this?

I think it is a safe to assume if Sparkleformation changes the behaviour, it will be a major version change, and we will need to revisit stack_master anyway 🤷‍♂️

[patrobinson]

If Sparkleformation fixes it at some time in the future, will we revert this?

Yes, I think it's best to follow the SparkleFormation documented golden path where possible