From 48e2eda62a2e31bb59230a878c7f87e1f600184a Mon Sep 17 00:00:00 2001 From: Manuel Kaufmann Date: Tue, 25 Jun 2019 16:15:37 +0200 Subject: [PATCH 1/7] Re-work around design document for APIv3 This commit, * describes main Goals/Non-Goals * includes implementation plan based on stages * updates the document mentioning what's already implemented (Version 1) * proposes a plan for iteration 2, 3 and 4 * mentions open questions for further discussion * includes potential feature out of the current roadmap --- docs/development/design/apiv3.rst | 233 ++++++++++++------------------ 1 file changed, 92 insertions(+), 141 deletions(-) diff --git a/docs/development/design/apiv3.rst b/docs/development/design/apiv3.rst index 8d1402b9d77..52edebe12fa 100644 --- a/docs/development/design/apiv3.rst +++ b/docs/development/design/apiv3.rst @@ -1,5 +1,8 @@ -Design of APIv3 -=============== +APIv3 +===== + +This document describes the design, +some decisions already made and an implementation plan for APIv3 in stages. APIv3 will be designed to be easy to use and useful to perform read and write operations as the main two goals. @@ -7,9 +10,25 @@ It will be based on Resources as APIv2 but considering the ``Project`` resource from where most of the endpoint will be based on it. +Goals +----- + +* Easy to use for our users +* Useful to perform read and write operations +* Cover most useful cases + + +Non-Goals +--------- + +* Filter by arbitrary and non-useful fields +* Cover *all the actions* available from the WebUI + + Problems with APIv2 ------------------- +There are several problem with our current APIv2 that we can list: * No authentication * It's read-only @@ -20,185 +39,117 @@ Problems with APIv2 * Footer API endpoint returns HTML -Proposed improves -+++++++++++++++++ - - -Use authentication -~~~~~~~~~~~~~~~~~~ - -* Pros: - - * queries can be personalized depending on the user - * allows us to perform write actions - -* Cons: - - * harder to implement - * requires a lot of time for good testing and QA - - -Questions: - -#. Do we want make auth a requirement? -#. Should we expose some endpoints as Read Only if not authenticated? -#. How we do communicate users about deprecation if they are Anonymous? Is it enough to add a note in the docs saying that "Auth is preferred for communication"? -#. How are we going to expose the ``Token`` required for Auth when building if auth is mandatory? - - -Read and Write -~~~~~~~~~~~~~~ - -* Pros: - - * most of the actions can be performed by a script instead of by - accessing readthedocs.org with a browser - -* Cons: - - * we have to deal with authorization - * open the door to possibilities of security holes +Implementation stages +--------------------- +Version 1 ++++++++++ -Questions: +The first implementation of APIv3 will cover the following aspects: -#. Do we want to allow the user to perform **all the write actions** - that are possible to do from the dashboard via the API? +.. note:: + This is currently implemented and live. Although, it's only for internal testing. -Design it for slugs -~~~~~~~~~~~~~~~~~~~ +* Authentication -* Pros: + * all endpoints require authentication via ``Authorization:`` request header + * detail endpoints are available for all authenticated users + * only Project's maintainers can access listing endpoints + * personalized listing - * knowing the slug of your project (which is presented to the user) - you can perform all the actions or retrieve all the data +* Read and Write -* Cons: + * edit attributes from Version (only ``active`` and ``privacy_level``) + * trigger Build for a specific Version - * it will be a mixture between most of the endpoint using ``slug`` - and to retrieve details of a Build it will be an ``id`` +* Accessible by slug + * Projects are accessed by ``slug`` + * Versions are accessed by ``slug`` + * ``/projects/`` endpoint is the main one and all of the other are nested under it + * Builds are accessed by ``id``, as exception to this rule + * access all (active/non-active) Versions of a Project by ``slug`` + * get latest Build for a Project (and Version) by ``slug`` + * filter by relevant fields -Expose internal endpoints -~~~~~~~~~~~~~~~~~~~~~~~~~ +* Proper status codes to report errors -There are some endpoints that we are using internally like -``/api/v2/search/``, ``/api/v2/footer/`` and -``/api/v2/sustainability/``. +* Browse-able endpoints + * browse is allowed hitting ``/api/v3/projects/`` as starting point + * ability to navigate clicking on other resources under ``_links`` attribute -* Pros: +* Rate limited - * allow to build custom footer - * allow to build custom search autocomplete widget -* Cons: +Version 2 ++++++++++ - * n/a +Second iteration will polish issues found from the first step, +and add new endpoints to allow *import a project and configure it* +without the needed of using the WebUI as a main goal. -Questions: +After Version 2 is deployed, +we will invite users that reach us as beta testers to receive more feedback +and continue improving it by supporting more use cases. -#. Do we want to add ``/api/v2/sustainability/`` to APIv3 or that - should be part of the new Ad Server that we are building? +This iteration will include: +* Minor changes to fields returned in the objects +* Import Project endpoint +* Edit Project attributes ("Settings" and "Advanced settings-Global settings" in the WebUI) +* Trigger Build for default version +* Allow CRUD for Redirect, Environment Variables and Notifications (``WebHook`` and ``EmailHook``) +* Documentation -Proper status codes for error reporting -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Pros: - - * user knows what it's happening and can take decisions about it - -* Cons: - - * n/a - - -Relationship between API resources -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Pros: - - * browse-able API by accessing to the default resource (Project) - * knowing the project's slug you can perform all the actions related to it - -* Cons: - - * more data is returned with all the links to the endpoints - - -Make footer API returns JSON -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Pros: - - * users can build their own version menu flyout - * cleaner than returning a HTML and injecting it - -* Cons: - - * we need to adapt our theme for this - - -Questions: - -#. Do we want an specific endpoint for the footer? -#. The flyout could be built by querying 2 or 3 of the regular - endpoint. Would this add too much overhead to the page loading - time? - - -Use cases ---------- -We want to cover -++++++++++++++++ +Version 3 ++++++++++ +Third iteration will implement granular permissions. +Keeping in mind how Sphinx extension will use it: -* Return all **my** projects -* Access all the resources by knowing the project's slug -* Ability to filter by fields +* ``sphinx-version-warning`` needs to get *all active Versions of a Project* +* An extension that creates a landing page, will need *all the subprojects of a Project* - * all the active versions of specific project +To fulfill these requirements, this iteration will include: -* Data about builds +* Scope-based authorization token - * latest build for project - * latest build for a particular version of a project - * status of a particular build -* Perform write actions like +Version 4 ++++++++++ - * add a Domain, - * add User as mantainer, - * import a new Project under my username, - * set the language of the Project, - * trigger a Build, - * activate/deactivate a Version to be built, - * and all the actions you can perform from the Admin tab. +* Specific endpoint for our flyout menu (returning JSON instead of HTML) -* Retrieve all the information needed to create a custom version menu flyout +Out of roadmap +-------------- -Considering some useful cases for the corporate site: +These are some features that we may want to build but they are not in the roadmap at this moment: -* Give access to a doc page (``objects.inv``, ``/design/core.html``) +* CRUD for Domain +* Add User as maintainer +* Give access to a documentation page (``objects.inv``, ``/design/core.html``) +* Internal Build process -We do NOT want to cover -+++++++++++++++++++++++ +Open questions +-------------- -* Random filtering over a whole and not useful Resource +There are some questions that we still have. +These will need more discussion before making a decision on where, +when and how to implement them. - * "All the ``stable`` versions" - * "Builds with ``exit_code`` equal to 257" +* Do we want to add ``/api/v2/sustainability/`` to APIv3? + Should be part of the new "Ad Server" that we are building? +* Should we make our search endpoint at ``/api/v2/search`` publicly on APIv3? -Technical aspects that would be good to have --------------------------------------------- +Nice to have +------------ -* Rate limit * ``Request-ID`` header * `JSON minified by default`_ (maybe with ``?pretty=true``) * `JSON schema and validation`_ with docs_ From a6e929a62eabc55bf3d497479dde535d9a2ae693 Mon Sep 17 00:00:00 2001 From: Manuel Kaufmann Date: Wed, 26 Jun 2019 18:31:13 +0200 Subject: [PATCH 2/7] Improve title --- docs/development/design/apiv3.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/development/design/apiv3.rst b/docs/development/design/apiv3.rst index 52edebe12fa..95069fd20a6 100644 --- a/docs/development/design/apiv3.rst +++ b/docs/development/design/apiv3.rst @@ -1,5 +1,6 @@ -APIv3 -===== +======================= + APIv3 Design Document +======================= This document describes the design, some decisions already made and an implementation plan for APIv3 in stages. From cd8c1635711cdf5b587bd3fe74442fb7b89d1be5 Mon Sep 17 00:00:00 2001 From: Manuel Kaufmann Date: Wed, 26 Jun 2019 18:31:20 +0200 Subject: [PATCH 3/7] Add table of contents --- docs/development/design/apiv3.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/development/design/apiv3.rst b/docs/development/design/apiv3.rst index 95069fd20a6..9c077d82f2b 100644 --- a/docs/development/design/apiv3.rst +++ b/docs/development/design/apiv3.rst @@ -10,6 +10,11 @@ APIv3 will be designed to be easy to use and useful to perform read and write op It will be based on Resources as APIv2 but considering the ``Project`` resource as the main one, from where most of the endpoint will be based on it. +.. contents:: + :local: + :backlinks: none + :depth: 1 + Goals ----- From 02ce6ecd8ae18eaf966c83995d7a64e3ac36d079 Mon Sep 17 00:00:00 2001 From: Manuel Kaufmann Date: Wed, 26 Jun 2019 18:31:42 +0200 Subject: [PATCH 4/7] Remove open questions We are not building this endpoints --- docs/development/design/apiv3.rst | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/docs/development/design/apiv3.rst b/docs/development/design/apiv3.rst index 9c077d82f2b..69d6252213b 100644 --- a/docs/development/design/apiv3.rst +++ b/docs/development/design/apiv3.rst @@ -141,18 +141,6 @@ These are some features that we may want to build but they are not in the roadma * Internal Build process -Open questions --------------- - -There are some questions that we still have. -These will need more discussion before making a decision on where, -when and how to implement them. - -* Do we want to add ``/api/v2/sustainability/`` to APIv3? - Should be part of the new "Ad Server" that we are building? -* Should we make our search endpoint at ``/api/v2/search`` publicly on APIv3? - - Nice to have ------------ From cb6c87ac5d8bf81f7e29d4ed166192b719311f48 Mon Sep 17 00:00:00 2001 From: Manuel Kaufmann Date: Wed, 26 Jun 2019 18:58:23 +0200 Subject: [PATCH 5/7] Explain more Out of roadmap --- docs/development/design/apiv3.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/development/design/apiv3.rst b/docs/development/design/apiv3.rst index 69d6252213b..af7b7fc3281 100644 --- a/docs/development/design/apiv3.rst +++ b/docs/development/design/apiv3.rst @@ -133,7 +133,9 @@ Version 4 Out of roadmap -------------- -These are some features that we may want to build but they are not in the roadmap at this moment: +These are some features that we may want to build at some point. +Although, they are currently out of our near roadmap because they don't affect too many users, +or are for internal usage only. * CRUD for Domain * Add User as maintainer From 71d07cf9d7fa2a951884d382226e29848394dfe8 Mon Sep 17 00:00:00 2001 From: Manuel Kaufmann Date: Wed, 26 Jun 2019 18:58:43 +0200 Subject: [PATCH 6/7] Expand introduction --- docs/development/design/apiv3.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/development/design/apiv3.rst b/docs/development/design/apiv3.rst index af7b7fc3281..3e3028bdcfe 100644 --- a/docs/development/design/apiv3.rst +++ b/docs/development/design/apiv3.rst @@ -3,7 +3,8 @@ ======================= This document describes the design, -some decisions already made and an implementation plan for APIv3 in stages. +some decisions already made and built (current Version 1 of APIv3) +and an implementation plan for next Versions of APIv3. APIv3 will be designed to be easy to use and useful to perform read and write operations as the main two goals. From 250ee1ffee1eaad1b9c35daaea596d690e68223a Mon Sep 17 00:00:00 2001 From: Manuel Kaufmann Date: Wed, 26 Jun 2019 18:58:55 +0200 Subject: [PATCH 7/7] Better Goals and Non-Goals --- docs/development/design/apiv3.rst | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/docs/development/design/apiv3.rst b/docs/development/design/apiv3.rst index 3e3028bdcfe..618023a2670 100644 --- a/docs/development/design/apiv3.rst +++ b/docs/development/design/apiv3.rst @@ -20,15 +20,31 @@ from where most of the endpoint will be based on it. Goals ----- -* Easy to use for our users +* Easy to use for our users (access most of resources by ``slug``) * Useful to perform read and write operations -* Cover most useful cases +* Authentication/Authorization + + * Authentication based on scoped-tokens + * Handle Authorization nicely using an abstraction layer + +* Cover most useful cases: + + * Integration on CI (check build status, trigger new build, etc) + * Usage from public Sphinx/MkDocs extensions + * Allow creation of flyout menu client-side + * Simplify migration from other services (import projects, create multiple redirects, etc) Non-Goals --------- * Filter by arbitrary and non-useful fields + + * "Builds with ``exit_code=1``" + * "Builds containing ``ERROR`` on their output" + * "Projects created after X datetime" + * "Versions with tag ``python``" + * Cover *all the actions* available from the WebUI