docs: should add instructions to run from source

[SgtPooki]

What I did

make gateway-conformance
go run ./cmd/gateway-conformance/main.go test # ...args

If there's a better way please let me know.

Also of note.. you cannot go install this package, but it would make testing much easier..

╰─ ✘ 1 ❯ go version
go version go1.21.4 darwin/arm64

╰─ ✘ 1 ❯ go install github.com/ipfs/gateway-conformance@latest
go: downloading github.com/ipfs/gateway-conformance v0.4.2
go: github.com/ipfs/gateway-conformance@latest: module github.com/ipfs/gateway-conformance@latest found (v0.4.2), but does not contain package github.com/ipfs/gateway-conformance

[SgtPooki]

go run github.com/SgtPooki/gateway-conformance@1f9d1f393d572b863de9b6303cd551573ac77a4c

🤷

[laurentsenta]

@SgtPooki, thanks for opening this, could you try going through the local dev documentation and detail which parts are not working for you?

I don't understand the meaning of your last comment.

Regarding making the testing easier, the docker image might be a better place to start. It abstracts away all the implementation details for users who don't want to look into go-related details. docs

[SgtPooki]

@SgtPooki, thanks for opening this, could you try going through the local dev documentation and detail which parts are not working for you?

Will do.

I don't understand the meaning of your last comment.

@laurentsenta are you referring to:

Also of note.. you cannot go install this package, but it would make testing much easier..

I was hoping to be able to execute gateway-conformance as a binary with something like go install github.com/ipfs/gateway-conformance and then gateway-conformance <subcmd>

[SgtPooki]

Some errors when following instructions at https://github.com/ipfs/gateway-conformance/blob/main/docs/development.md

╰─ ✔ ❯ make provision-kubo
find ./fixtures -name '*.car' -exec ipfs dag import --stats --pin-roots=false {} \;
Imported 3 blocks (119 bytes)
Imported 9 blocks (1541 bytes)
Imported 2 blocks (128 bytes)
Imported 11 blocks (759 bytes)
Imported 9 blocks (1462 bytes)
Imported 10 blocks (1538 bytes)
Imported 9 blocks (1541 bytes)
Imported 4 blocks (208 bytes)
Imported 243 blocks (74982 bytes)
Imported 3 blocks (2215 bytes)
Imported 10 blocks (623 bytes)
Imported 5 blocks (224 bytes)
Imported 1 blocks (26 bytes)
Imported 1 blocks (44 bytes)
Imported 3 blocks (148 bytes)
Imported 4 blocks (184 bytes)
Imported 11 blocks (711 bytes)
Imported 1 blocks (52 bytes)
Imported 3 blocks (231 bytes)
Imported 1 blocks (17 bytes)
Imported 3 blocks (139 bytes)
Imported 4 blocks (176 bytes)
Imported 10 blocks (623 bytes)
Imported 2 blocks (115 bytes)
Imported 32 blocks (68071 bytes)
Imported 3 blocks (189 bytes)
find ./fixtures -name '*.ipns-record' -exec sh -c 'ipfs routing put --allow-offline /ipns/$(basename -s .ipns-record "{}") "{}"' \;
12D3KooWLQzUv2FHWGVPXTXSZpdHs7oHbXub2G5WC8Tx4NQhyd2d
QmVujd5Vb7moysJj8itnGufN7MEtPRCNHkKpNuA4onsRa3
12D3KooWRL2ppK37M2ih6xGfufWVa15jqAcgeQ8ZvoAUJFAfbueZ
12D3KooWDUyFJTGJADckWJdfh2haqj6ckaXFWi1DReJ4RP8KsHg5
12D3KooWAe38Bk28zHVKiRHUhkjVmPAU9r4NoK5sxffK7EkHgHcU
Error: failed to parse peer ID: invalid cid: invalid base36 character ('_')
Error: failed to parse peer ID: invalid cid: invalid base36 character ('_')
Error: failed to parse peer ID: invalid cid: invalid base36 character ('_')
Error: failed to parse peer ID: invalid cid: invalid base36 character ('_')
Error: failed to parse peer ID: invalid cid: invalid base36 character ('_')
Error: failed to parse peer ID: invalid cid: invalid base36 character ('_')

╰─ ✔ ❯ ipfs --version
ipfs version 0.24.0

And from running the tests in vscode (which is great btw, thanks for directing me to that setup):

# Running TestGatewaySubdomains from vscode:
Running tool: /Users/sgtpooki/.asdf/installs/golang/1.20.6/go/bin/go test -timeout 30s -run ^TestGatewaySubdomains$ github.com/ipfs/gateway-conformance/tests

2023-12-05T08:54:58.023-0800    DEBUG   conformance     test/config.go:50       GatewayURL: http://127.0.0.1:8090
2023-12-05T08:54:58.023-0800    DEBUG   conformance     test/config.go:52       SubdomainGatewayURL: helia-http-gateway.localhost
2023-12-05T08:54:58.023-0800    DEBUG   conformance     test/config.go:53       SubdomainGatewayHost:
2023-12-05T08:54:58.023-0800    DEBUG   conformance     test/config.go:54       SubdomainGatewayScheme:
=== RUN   TestGatewaySubdomains
    /Users/sgtpooki/code/work/protocol.ai/ipfs/gateway-conformance/tests/subdomain_gateway_ipfs_test.go:111: --- META: {"group":"Subdomains"}
    /Users/sgtpooki/code/work/protocol.ai/ipfs/gateway-conformance/tests/subdomain_gateway_ipfs_test.go:363: parse "://bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am.ipfs.": missing protocol scheme
--- FAIL: TestGatewaySubdomains (0.00s)
FAIL
FAIL    github.com/ipfs/gateway-conformance/tests       0.223s

but running TestCors tests executes successfully

[laurentsenta]

I was hoping to be able to execute gateway-conformance as a binary with something like go install github.com/ipfs/gateway-conformance and then gateway-conformance

I believe that's where the docker image should be used instead. As a user, if I don't care about the test's internals, having a workflow with go install or docker might fill the same need. docs

find ./fixtures -name '*.ipns-record' -exec sh -c 'ipfs routing put --allow-offline /ipns/$(basename -s .ipns-record "{}") "{}"' ;
...
Error: failed to parse peer ID: invalid cid: invalid base36 character ('_')

That code is calling ipfs routing put --allow-offline /ipns/.... to provision your local instance. From what I can see in CI, it's supposed to work (Provision Gateway Step).

I can't dig into this at the moment, but an idea:
in the Makefile, add echo in front of the ipfs routing command and check which files are failing and try to run the command by hand. Maybe with verbose mode or debug. (I had success using export GOLOG_LOG_LEVEL=DEBUG before). @hacdias worked on ipns records, he might have an idea as well.

2023-12-05T08:54:58.023-0800 DEBUG conformance test/config.go:52 SubdomainGatewayURL: Helia-HTTP-gateway.localhost
2023-12-05T08:54:58.023-0800 DEBUG conformance test/config.go:53 SubdomainGatewayHost:
2023-12-05T08:54:58.023-0800 DEBUG conformance test/config.go:54 SubdomainGatewayScheme:
...
/Users/sgtpooki/code/work/protocol.ai/ipfs/gateway-conformance/tests/subdomain_gateway_ipfs_test.go:363: parse "://bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am.ipfs.": missing protocol scheme

It looks like we passed a string that is not a URL; the host and scheme are all messed up during parsing.
Could you try passing https://helia-http-gateway.localhost as your subdomain gateway URL? (we'll have to patch the test suite to panic when that happens).

TestCors doesn't rely on subdomain spec, so that checks out.

Thanks for going through & reporting all these quirks 🙏

[hacdias]

Regarding go install: I think you have to install the command, not the package itself. Therefore:

go install github.com/ipfs/gateway-conformance/cmd/gateway-conformance@latest

That should work. As @laurentsenta pointed out, it's mostly used with the Docker image though.

[hacdias]

#192 fixes make provision-kubo

[SgtPooki]

go install github.com/ipfs/gateway-conformance/cmd/gateway-conformance@latest

This works, and then I can run with $HOME/go/bin/gateway-conformance test