An experimental validator for OGC EDR APIs using schemathesis.
Team Punkt at met.no
Experiment - Every error reported might as well be an error in the validator, rather than the API.
Run docker image:
docker run -it --rm ghcr.io/metno/sedr --url https://edrisobaric.k8s.met.no
Debug logging will show every request and it's status:
- Run container, mounting the current directory at /logs, and tell container to output log there:
docker run -it --rm -v .:/logs ghcr.io/metno/sedr --openapi https://edrisobaric.k8s.met.no/api --url https://edrisobaric.k8s.met.no --log-file /logs/debug.log
Run manually as noted in Test it out, or add it to your CI using one of these examples:
- Assuming Openapi 3.1
- Assuming OGC EDR API version 1.2 (draft)
- Few, basic tests for now
- Will focus more on profiles (limitations within the EDR spec) like https://github.com/EURODEO/rodeo-edr-profile than the full EDR spec.
- Use --rodeo-profile to force a test against the profile (will happen automatically if conformance includes the profile)
For each "FAILED" line, you can scroll back to see the full error and, if relevant, with a curl-example to reproduce it.
================================ short test summary info =================================
ERROR sedr/schemat.py::test_landingpage - Failed: Test function sedr/schemat.py::test_landingpage does not match any API operat...
ERROR sedr/schemat.py::test_conformance - Failed: Test function sedr/schemat.py::test_conformance does not match any API operat...
!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!
=================================== 2 errors in 3.51s ====================================
================================ short test summary info =================================
ERROR sedr/schemat.py - schemathesis.exceptions.SchemaError: Failed to load schema due to client error (HTTP ...
!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!
==================================== 1 error in 2.94s ====================================
FAILED sedr/schemat.py::test_api[GET /] - requests.exceptions.ConnectionError: HTTPConnectionPool(host='example.com', port=80): M...
================================ short test summary info =================================
ERROR sedr/schemat.py - schemathesis.exceptions.SchemaError: Connection failed
!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!
==================================== 1 error in 3.52s ====================================
E schemathesis.exceptions.SchemaError: Failed to load schema due to client error (HTTP 404 Not Found)
================================ short test summary info =================================
ERROR sedr/schemat.py - schemathesis.exceptions.SchemaError: Failed to load schema due to client error (HTTP ...
!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!
==================================== 1 error in 3.64s ====================================
Sedr supports EDR 1.1, but the API is EDR 1.0.
if not requirementA2_2_A5:
> raise AssertionError(requirementA2_2_A5_message)
E AssertionError: Conformance page /conformance does not contain the core edr class http://www.opengis.net/spec/ogcapi-edr-1/1.1/conf/core. See <https://docs.ogc.org/is/19-086r6/19-086r6.html#_c9401fee-54b9-d116-8365-af0f85a8243d> for more info.
sedr/schemat.py:123: AssertionError
========================================= PASSES =========================================
____________________________________ test_api[GET /] _____________________________________
_______________________________ test_api[GET /conformance] _______________________________
_______________________________ test_api[GET /collections] _______________________________
________________________ test_api[GET /collections/observations] _________________________
___________________ test_api[GET /collections/observations/locations] ____________________
____________ test_api[GET /collections/observations/locations/{location_id}] _____________
____________________ test_api[GET /collections/observations/position] ____________________
______________________ test_api[GET /collections/observations/area] ______________________
_____________________ test_api[GET /collections/observations/items] ______________________
________________ test_api[GET /collections/observations/items/{item_id}] _________________
________________________________ test_landingpage[GET /] _________________________________
___________________________ test_collections[GET /collections] ___________________________
================================ short test summary info =================================
PASSED sedr/schemat.py::test_api[GET /]
PASSED sedr/schemat.py::test_api[GET /conformance]
PASSED sedr/schemat.py::test_api[GET /collections]
PASSED sedr/schemat.py::test_api[GET /collections/observations]
PASSED sedr/schemat.py::test_api[GET /collections/observations/locations]
PASSED sedr/schemat.py::test_api[GET /collections/observations/locations/{location_id}]
PASSED sedr/schemat.py::test_api[GET /collections/observations/position]
PASSED sedr/schemat.py::test_api[GET /collections/observations/area]
PASSED sedr/schemat.py::test_api[GET /collections/observations/items]
PASSED sedr/schemat.py::test_api[GET /collections/observations/items/{item_id}]
PASSED sedr/schemat.py::test_landingpage[GET /]
PASSED sedr/schemat.py::test_collections[GET /]
PASSED sedr/schemat.py::test_collections[GET /conformance]
PASSED sedr/schemat.py::test_collections[GET /collections]
PASSED sedr/schemat.py::test_collections[GET /collections/observations]
PASSED sedr/schemat.py::test_collections[GET /collections/observations/locations]
PASSED sedr/schemat.py::test_collections[GET /collections/observations/locations/{location_id}]
PASSED sedr/schemat.py::test_collections[GET /collections/observations/position]
PASSED sedr/schemat.py::test_collections[GET /collections/observations/area]
PASSED sedr/schemat.py::test_collections[GET /collections/observations/items]
PASSED sedr/schemat.py::test_collections[GET /collections/observations/items/{item_id}]
FAILED sedr/schemat.py::test_conformance[GET /conformance] - AssertionError: Conformance page /conformance does not contain the core edr class htt...
======================== 1 failed, 21 passed in 95.89s (0:01:35) =========================
Main components of the validator are:
Create an issue or start a discussion. Please do not contriute without discussing it first.