dist: Add distrotest lib

[phlax]

Signed-off-by: Ryan Northey ryan@synca.io

Commit Message: dist: Add distrotest lib
Additional Description:

This is a breakout from #17380 and #16899

It adds a a lib that for given arguments:

• creates a docker image (based on a distro - eg debian/ubuntu/redhat)
• starts a container with the image
• runs tests against provided installables - ie debs/rpms

this doesnt include the Checker to run the tests in ci etc

theres quite a bit of code in #17380 so i figured to break the lib/tests out first

this will also allow tweaking the cli args in the checker once this has landed

Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]

[phlax]

coverage is here https://storage.googleapis.com/envoy-pr/17488/tooling/index.html

[phlax]

cc @htuch

[htuch]

Looks really clean, just some minor comments/nits.
/wait

[htuch]

Did you consider modeling the configuration format as proto3 (with YAML frontend) to provide a schema/documentation?

[phlax]

the idea crossed my mind, but tbh i didnt see the immediate benefit.

I guess though, as you say, the benefit is documentation and a schema to work with

there are 2 yaml configs

this one defines the build/exec environment for the different container types - ie deb/rpm - its more complex but unlikely to change much and is not public-facing/configurable, at least with current implementation of the checker.

the other yaml schema (which comes as part of the next pr) is configurable/public-facing, but is super simple - basically just

debian_buster:
    image: debian:buster-slim
    ext: buster.changes

i would be happy following up to add protobuf schemas for these if you were ok with that

[phlax]

the downside is i guess the dependency on protobuf

[htuch]

Up to you, just thought it might have some win. My only strongly held opinion is that if we go with a schema, it should be protobuf, to reduce the proliferation of schema languages/tooling. Protobuf dependency isn't too bad I think, we have this in a fair bit of Python tooling already.

[phlax]

yep, noted

[phlax]

i think its theoretically possible to derive the maintainer from the keyfile but probs i would prefer to follow up with this as it adds some complexity

[htuch]

LGTM, but I think you have some changes still in flight so can approve again later. I didn't review the unit tests in depth.

[htuch]

LGTM, thanks!