Run tests using reference data from purl-spec

[keshav-space]

Adds test coverage using reference data provided by purl-spec https://github.com/aboutcode-org/purl-spec/tree/main/tests/types.
The test schema is available https://github.com/aboutcode-org/purl-spec/blob/main/schemas/purl-test.schema.json.

Note

Out of 481 reference tests, 85 are failing.

Related issue: aboutcode-org/purl-spec#30

[petergardfjall]

Out of 481 reference tests, 85 are failing.

Hey, thanks for doing this. I started looking at the failing tests and hope to be able to put up a PR soon.
I did however run into, what I think are, problems with the test files (package-url/purl-spec#644, package-url/purl-spec#643).

But there is indeed a number of fixes needed to the code, not least is it not very good at encoding purls according to spec.

I based my code on your changes in this PR and have a few suggestions which I felt made life simpler when writing (and trying to fix broken) tests.

Would you mind me commenting although I'm no repo maintainer?

[keshav-space]

Would you mind me commenting although I'm no repo maintainer?

Yes please, suggestions are welcome.

[petergardfjall]

I've proposed a few changes which made the tests a bit easier to work with for me at least. Feel free to incorporate them!

[petergardfjall]

Hey, for you information I've opened a PR where I attempt to fix the code according to spec: #83

It still fails on a number of the new test cases but I've opened a few tickets since I don't understand the requirements that the test cases appear to place on library implementations:

• Expected output in testcases do not follow spec encoding purl-spec#644
• Inconsistent test cases for Conan purl-spec#643
• Incorrect test case for pkg:maven purl-spec#645

[keshav-space]

@petergardfjall thanks, I've incorporated your suggestion into the PR.

[keshav-space]

It still fails on a number of the new test cases but I've opened a few tickets since I don't understand the requirements that the test cases appear to place on library implementations:

I might be wrong, but I think there was an in-flight PR by @TG1999 to fix the encoding related test in the purl-spec repo. I can't seem to find it.

[pombredanne]

@petergardfjall do you want to get committer access?

[petergardfjall]

@petergardfjall do you want to get committer access?

Hey @pombredanne! What does "committer access" mean in practice here?
Does it mean "maintainer". I can't say that I know all the ins and outs of the purl spec (although I try as you may have seen from my recent tickets about questions on the spec test case 😅 ).

I'm not sure I would be comfortable making commits without someone knowledgable to review them.

The purl spec is very important in the project I'm currently working on so quicker turnaround/responsiveness is desriable.
But not at the expense of correctness. 😂

[pombredanne]

@petergardfjall this would give you commit rights, but we are not committing things without reviews for sure!
RFC posted at #84

[pombredanne]

@shibumi @mcombuechen we need to adopt the new test suite here soon enough... any objection to this PR? Can review and approve? Next up will be to fix the tests.

[shibumi]

SGTM to me. We just have to fix the tests :)

[petergardfjall]

I think this is good to go. Would another maintainer care to review?

With the code changes in #83 (using packageurl_test.go from this branch) and some needed updates to the purl-spec test suite I think the library is in a good (better) state.

Merging this PR first and then merging #83 (after review, of course) seems like a sensible course of action to me.

[petergardfjall]

Giving it my approval in case that helps.

[keshav-space]

@petergardfjall thanks for the encoding fix in the purl-spec test fixture. Number of failing tests is now 65, down from 85.

[petergardfjall]

I've rebased #83 onto this PR and all tests now pass (if pending purl-spec testcase PRs are accepted)!

So merging #80 into master brings these commits with it.

[shibumi]

Unit tests are failing. Can you take a look?

[pombredanne]

Thanks. Merging now!