refactor(icon-build-helpers): update metadata generation

[joshblack]

This PR closes several long-standing issues we've had with our metadata generation as consumed by the website group. In particular:

• The underlying metadata.yml is no longer nested and is now a flat file
• There is no longer duplicate data between YML files
• Adds support for authoring information about assets in a pluggable way
• Fields that the website group had to compute are now provided out-of-the-box

The majority of this diff is consumed by the generation of pictograms.yml and icons.yml files, feel free to ignore them! In addition, changes in this PR should have no impact on public API of our icons or component packages so changes should be reserved to internal-only build tooling.

Feel free to reach out if you would like a quick run-through of the PR! There is also a README with a quick explainer over here.

The main interface for working with these changes is discoverable through packages/icons/tasks/{build,ci-check,scaffold}.js which use the newly created packages/icon-build-helpers/src/metadata/index.js entrypoint.

The most common use-cases include:

• build: generate a metadata.json file to be consumed by the website. This file is generated from several data sources, including icon, category, deprecation, and module name info
• ci-check: verify that all assets in the svg folder match what is written in metadata, and vice-versa
• scaffold: help scaffold out metadata information for designers contributing iconography

These commands are powered by some of the following concepts (also explained in the README!)

• We build up a "registry" of available SVG assets defined in a source directory
  • An icon can have multiple assets, for example switcher may have an asset for 16, 20, 24, and 32px (although this can change)
• Information about icons are written in YML files and split out. These are called "extensions"
  • This helps with authoring data about a large number of icons
  • The file format (YML) can be swapped out through our usage of "adapters"
• We validate the data in extensions that are written with the information made available through the registry
  • For example, if we have an icon in metadata we expect it to be in the registry. If an icon is in the registry, we expect it to have metadata
• The output metadata is generated through chaining extensions together which each modify the metadata with certain bits of information. For example:
  • Icons will add an icons array with all the icons available
  • Deprecated will add a deprecated field on deprecated icons
  • Categories will add category information for each icon
  • Module name will add the module name for the asset in code

Changelog

New

• Add Metadata module to icon-build-helpers that powers building, checking, and scaffolding metadata information about icons and pictograms
• metadata/migrations folder that hosts migration scripts that are run to convert one metadata file format to another

Changed

• Migrate nested metadata.yml files to flat icons.yml and pictograms.yml files

Removed

[netlify]

Deploy preview for carbon-elements ready!

Built with commit d08fc30

https://deploy-preview-5372--carbon-elements.netlify.com

[netlify]

Deploy preview for carbon-components-react ready!

Built with commit d08fc30

https://deploy-preview-5372--carbon-components-react.netlify.com

[joshblack]

cc @vpicone!

[dakahn]

Seems real cool -- thanks for walking me through this earlier today @joshblack

[asudoh]

Thank you for the work @joshblack - Great if the new file format for each .yml files are documented somewhere, if it's not done yet.

[joshblack]

@asudoh agreed! Each extension should have a corresponding schema field that will validate the file format:

carbon/packages/icon-build-helpers/src/metadata/extensions/icons.js

Lines 24 to 34 in 638b72c

	schema: Joi.array().items(
	Joi.object().keys({
	name: Joi.string().required(),
	friendly_name: Joi.string().required(),
	sizes: Joi.array().items(
	Joi.string().valid('glyph'),
	Joi.number().valid([16, 20, 24, 32])
	),
	aliases: Joi.array(),
	})
	),

This structure corresponds to what we receive after the YML file is transformed into JS 👍

[tw15egan]

LGTM 👍 ✅

Thanks a lot for the detailed PR / README file!