Test that our packages can be successfully installed

[luhring]

We should verify that an apk client (e.g. the apk command, apko, others?) can successfully install every package we produce. This should be a check in CI, and ideally it'd be easy to run locally, too.

This would cover:

1. issues with post-install scripts (called out specifically here: Add CI check to verify that post-install scripts work correctly #418)
2. problems installing runtime dependencies
3. other things?

[rawlingsj]

FWIW I had a little experiment a while back that did this in a GitHub action, something like this

  package-test-job:
    name: Verify Package
    needs: [build]
    runs-on: ubuntu-latest
    container:
      # image: cgr.dev/chainguard/wolfi-base:latest
      image: rawlingsj80/wolfi-test:latest
    steps:
    - name: Retrieve apks
      uses: actions/download-artifact@v3
      with:
        name: packages-artifact
        path: /work/packages
    - name: Retrieve temporary public signing key
      uses: actions/download-artifact@v3
      with:
        name: public-key-artifact
        path: /work

    - name: Prepare local apk repository
      run: |
        cp /work/local-melange.rsa.pub /etc/apk/keys/
        echo "" >> /etc/apk/repositories
        echo "/work/packages" >> /etc/apk/repositories

    - name: Add package
      run: apk add foo

    - name: Test package
      run: cat /usr/foo | grep foo

    - name: Scan Packages
      run: grype . --file grype_scan.md

    - name: PR comment with scan results
      uses: thollander/actions-comment-pull-request@v2
      with:
        filePath: grype_scan.md

That also uploaded the scan results as a comment to the PR.

@kaniini suggested at the time we could add a test pipeline to melange that ran the tests instead which would be good. Maybe we could even use a uses: image://cgr.dev/chainguard/wolfi-base:latest similar to github to describe which image to use for the test.

Commenting in case any of this is useful.