Implementation of APIv3 #4863

humitos · 2018-11-05T10:46:34Z

Initial ideas to discuss the implementation of APIv3.

This is a work in progress PR that only has some pieces of the documentation with the idea to start a discussion around the general proposal.

Closes #4286

humitos

Some thoughts on the overall idea but also where write endpoints make sense.

humitos · 2018-11-05T10:53:13Z

docs/api/v3.rst

+~~~~~~
+
+The API can be accessed by using the HTTP header ``Authorization``
+with a specific ``Token`` that provide access to different resources.


We may probably allow a Cookie session also here.

@agjohnson has a point about not supporting this that we should consider. He wrote something at #5009 (comment)

humitos · 2018-11-05T10:57:32Z

docs/api/v3.rst

+and do require authentication even for requesting public information.
+
+Scopes
+~~~~~~


I need to keep thinking about the implementation of this, since given a Token the request won't be authenticated under a particular user but as a "token with limited permissions".

So, queryset for API endpoints will need to manage these two cases:

projects where the user is admin (in case we support cookie sessions)

projects where that token has specific permissions

Why don't we want tokens tied to a user? A user might have tokens that have less than full permissions but I think we definitely want tokens tied to a user. Maybe I'm misunderstanding this.

You are right. I didn't express myself properly. The 1) case is the same for cookie sessions than for a token user (without custom permission scopes).

Summarizing, we will have these authentication methods:

regular session cookie (authenticated as a user)

token tied to a user (authenticated as a user)

token with specific permissions (not tied to a user)

it could be project specific: project:read only in docs project (multiple projects also?)

it could be all project from a user specific: project:read for all the projects belonging to user

humitos · 2018-11-05T11:01:10Z

docs/api/v3.rst

+     * repo type
+     * version active
+     * version built
+     * all database?


I may want to perform a query like

/api/v3/project/?repo_url=https://github.com/pypa/pip

and get all the projects that are using that URL as source.

Or, all the Spanish projects under Read the Docs,

/api/v3/project/?language=es

but I don't want to allow listing all the projects without any kind of filtering.

We need to be very careful on this. For projects (where we have ~100k) this might be ok. For builds or versions, we should probably only allow filtering on indexed fields (eg. slug, project__slug, etc.). Otherwise, this can result in API calls that are very heavy from a performance and load perspective.

humitos · 2018-11-05T11:02:06Z

docs/api/v3.rst

+Project list
++++++++++++
+
+.. http:get::  /api/v3/project/


This should probably be called /api/v3/project/me/ or similar so we can allow some global filtering now or in the future but leaving the feature open.

humitos · 2018-11-05T11:03:36Z

docs/api/v3.rst

+                {
+                    "href": "/api/v3/project/pip/users/",
+                    "rel": "users",
+                    "type": "GET"


I find this really verbose but very useful to discover the API and navigate over it.

I suppose that if we really do care about the amount of data returned here, we should re-consider GraphQL ;)

I mean, you know the slug of your project, you hit this endpoint and you know everything you can do with the API and your project.

Shouldn't type be method to be more precisely named? However, I question whether it is even needed. If you are just listing, it is always a GET.

href, rel and type are the "standards" (or what most of the docs I read use).

I think we could remove it completely as you say, but I think that POST would also be allowed in this particular endpoint to add a Maintainer/Owner for this particular project for example.

humitos · 2018-11-05T11:05:45Z

docs/api/v3.rst

+
+.. TODO:
+
+   Add query string filters to narrow the query:


?featured=true could be useful for the home page also.

humitos · 2018-11-05T11:08:20Z

docs/api/v3.rst

+                    "type": "GET"
+                },
+                {
+                    "href": "/api/v3/project/pip/builds/",


I'm thinking that you can do a POST to this URL with something like:

{ "version": "latest" }

and trigger a build for that project and version.

and probably similar POST request for the rest of the endpoints listed here as links and GET.

/domains could add a new domain, /notifications a new webhook or email notification, etc

humitos · 2018-11-05T11:14:27Z

docs/api/v3.rst

+Version detail
++++++++++++++
+
+.. http:get::  /api/v2/project/(string:project-slug)/version/(string:version-slug)/


By POSTing here, we could activate or disable the version, change the privacy level, etc

humitos · 2018-11-05T11:23:33Z

docs/api/v3.rst

+            "default_version": "stable",
+            "default_branch": "master",
+            "documentation_type": "sphinx_htmldir",
+            "canonical_url": "http://pip.pypa.io/en/stable/",


tags are missing

humitos · 2018-11-05T11:54:56Z

docs/api/v3.rst

+Project details
+++++++++++++++
+
+.. http:get::  /api/v3/project/(string:slug)/


Everything should be accessed by string:slug or id:int.

https://geemus.gitbooks.io/http-api-design/content/en/requests/support-non-id-dereferencing-for-convenience.html

Do we really want/need this at all?

We are currently accessing by id to the different endpoint from projects/tasks.py from builders, but I think that this was built like this because there wasn't another way.

I'm not sure that accessing by ID is useful in our case.

I don't think we need to expose the ID.

👍 to not exposing ID. slug is the public facing unique identifier in my mind. Though it is possible that it could change over time (by an admin), but I think that's an edge case.

humitos · 2018-11-05T12:22:44Z

docs/api/v3.rst

+                "htmlzip": "//readthedocs.org/projects/pip/downloads/htmlzip/stable/",
+                "epub": "//readthedocs.org/projects/pip/downloads/epub/stable/"
+            },
+            "links": [


To build this structure, we could use something similar to https://www.django-rest-framework.org/tutorial/5-relationships-and-hyperlinked-apis/#hyperlinking-our-api

I personally like the idea of links to show related endpoints to the current resource (and to get more detailed info about it) but the list structure does not convince me since if you want the builds url, you need to loop over the whole list. Maybe a dict is better.

On the other hand, using a dict will also require another nested dict for the method type, like:

"links": { "builds": { "GET": { "href": URL, ... }

I'm not 100% convinced on the structure, but I think it's a good adoption to have these HATEOAS links here.

Well... as there is not only one solution for this, I kept googling it and I found that what I'm suggesting here is something already adopted: https://slides.com/jcassee/pycon-sette-hypermedia#/2/6

This is the structure:

links: rel: href: URI

humitos · 2018-11-05T13:01:39Z

docs/api/v3.rst

+        {
+            "id": 7367364,
+            "date": "2018-06-19T15:15:59.135894",
+            "length": 59,


I think this field should be called duration

humitos · 2018-11-05T13:09:01Z

docs/api/v3.rst

+                    "type": "GET"
+                },
+                {
+                    "href": "/api/v3/project/pip/notifications/",


All of these URLs can be expressed in the other way:

/api/v3/notifications/pip/

but for some endpoints doesn't look great. Example,

/api/v3/versions/pip/stable/

I'm not 100% on either, though. In this way it's clear that you will be returned a Version resource and /api/v3/project/pip/versions/stable/ maybe it's not that clear.

This document says something about this: https://geemus.gitbooks.io/http-api-design/content/en/requests/minimize-path-nesting.html

Another comment here is that we are using a similar approach of nested URLs for the URLs site: https://readthedocs.org/dashboard/(project-slug)/version/(version-slug)/

I would try hard to not have multiple endpoints that return essentially the same thing. I think that point is not that all API nesting is bad but more that it can go too far.

I prefer /api/v3/project/pip/notifications/ as an endpoint to /api/v3/notifications/pip/ (I would prefer /api/v3/notifications/?project__slug=pip to the 2nd one although not the 1st). Since I think /api/v3/notifications/ -- notifications for all projects -- is not particularly useful I think it's fine to only expose the endpoint that is specific to a project.

humitos · 2018-11-05T15:25:35Z

docs/api/v3.rst

+Build detail
++++++++++++
+
+.. http:get::  /api/v3/build/(int:id)/


In this case, it makes sense to access by id since it's only unique identifier that we have for a build.

This probably shoulnd't be allowed (since doesn't make to much sense as a whole) and instead we should make it like /api/v3/project/pip/builds/?commit=a1b2c3 or ?latest=true or ?running=true.

stsewd · 2018-11-05T15:49:06Z

docs/api/v3.rst

+Project list
++++++++++++
+
+.. http:get::  /api/v3/project/


I think we should start using the resource name in plural for v3, /projects/

Yes.

I made a mistake and then copy&paste, but it should be plural here.

humitos · 2018-11-05T16:16:46Z

docs/api/v3.rst

+            "slug": "stable",
+            "verbose_name": "stable",
+            "identifier": "3a6b3995c141c0888af6591a59240ba5db7d9914",
+            "built": true,


I'd like to have a calculated field here: build_date or last_success_build_date or something with that meaning but a better name ;)

davidfischer

I like where this is going.

davidfischer · 2018-11-05T16:59:41Z

docs/api/v3.rst

+and do require authentication even for requesting public information.
+
+Scopes
+~~~~~~


Why don't we want tokens tied to a user? A user might have tokens that have less than full permissions but I think we definitely want tokens tied to a user. Maybe I'm misunderstanding this.

davidfischer · 2018-11-05T17:01:40Z

docs/api/v3.rst

+     * repo type
+     * version active
+     * version built
+     * all database?


We need to be very careful on this. For projects (where we have ~100k) this might be ok. For builds or versions, we should probably only allow filtering on indexed fields (eg. slug, project__slug, etc.). Otherwise, this can result in API calls that are very heavy from a performance and load perspective.

davidfischer · 2018-11-05T17:02:25Z

docs/api/v3.rst

+Project details
+++++++++++++++
+
+.. http:get::  /api/v3/project/(string:slug)/


I don't think we need to expose the ID.

davidfischer · 2018-11-05T17:07:41Z

docs/api/v3.rst

+                    "type": "GET"
+                },
+                {
+                    "href": "/api/v3/project/pip/notifications/",


I would try hard to not have multiple endpoints that return essentially the same thing. I think that point is not that all API nesting is bad but more that it can go too far.

I prefer /api/v3/project/pip/notifications/ as an endpoint to /api/v3/notifications/pip/ (I would prefer /api/v3/notifications/?project__slug=pip to the 2nd one although not the 1st). Since I think /api/v3/notifications/ -- notifications for all projects -- is not particularly useful I think it's fine to only expose the endpoint that is specific to a project.

davidfischer · 2018-11-05T17:10:43Z

docs/api/v3.rst

+                {
+                    "href": "/api/v3/project/pip/users/",
+                    "rel": "users",
+                    "type": "GET"


Shouldn't type be method to be more precisely named? However, I question whether it is even needed. If you are just listing, it is always a GET.

davidfischer · 2018-11-05T17:11:29Z

docs/api/v3.rst

+
+    :statuscode 201: Created sucessfully
+    :statuscode 400: Some field is invalid
+    :statuscode 401: Not valid permissions


I don't think this is necessary to list on every endpoint. This should probably just be in some section on authentication.

davidfischer · 2018-11-05T17:12:56Z

docs/api/v3.rst

+    :statuscode 401: Not valid permissions
+
+
+Footer


I think we should come up with a better name for this API endpoint than footer. Maybe displaymetadata. Maybe it can even go away because all the necessary data is in other endpoints. Not critical but something to think about.

I thought on removing it in favor of all the other endpoints but in that case to build the flyout you may need to make 2 or 3 or maybe more requests. Is that affordable regarding request time?

ericholscher

I like the direction that this is going. I think it could use a design document explaining some of the design decisions that went into it. I also think it would be useful to have some explicit use cases that we care about, and want to support and aren't supporting. It feels like they are implied here, but having them be explicit would be very useful. For example, this API feels very focused on automating dashboard operations, but not doing anything with documentation page features, which is arguably ~90% of what our users want to be doing w/ RTD.

ericholscher · 2018-11-05T17:50:32Z

docs/api/v3.rst

+--------------------------------
+
+Requests to the Read the Docs public API are for public and private information
+and do require authentication even for requesting public information.


I'm -0 on this. I think having a public API that isn't authed fits a lot of use cases nicely, and I haven't heard an argument for why we should exclude it in this design. I'm fine to heavily rate limit anon access so it isn't built into processes, but I think for learning etc. having it open is good.

Also, is it possible to use this API on a documentation page as designed? eg. I'm a user who wants to get the version listing of my project in my own JS, can I do that?

To me, not requiring authentication duplicates a mistake we made with API v1 and v2. Now that we are deprecating API v1 and v2, there's no way to know who is making outdated API calls because the calls aren't authenticated. Other than broadcast messages in our docs and on the blog, we can't warn users specifically that their API calls are going to break after a certain date.

Secondly, I question the usefulness of an API that allows browsing all projects, versions, and builds rather than "my" projects, versions, and builds. For example, our current v2 endpoints would be a lot more useful if they returned the few projects where you are a maintainer (at least by default) rather than /api/v2/projects/ returning all projects and requiring you to page through them or filter by a single slug.

Still, maybe the right approach is to heavily rate limit the API when used anonymously such that almost everyone will get a token.

Also, is it possible to use this API on a documentation page as designed? eg. I'm a user who wants to get the version listing of my project in my own JS, can I do that?

With authentication, probably not. However, how does one know which projects are "my" projects without authentication?

With authentication, probably not. However, how does one know which projects are "my" projects without authentication?

GET /api/v3/projects/<slug>/versions/

That gets a project but there's no concept of my project without auth.

well sure, but presumably the JS embedded in a project's docs know what the project is :)

I agree in general auth is useful, but I think we need to support unauth'd endpoints for a lot of pretty important use cases, or have some kind of publishable token.

Wait a second... I think Eric has a point. How are we going to use our own API (for the footer for example) if we do not allow anonymous access? Will we allow anonymous access only on the endpoints that we know that are going to be used for any doc page?

I'm not sure if we should raise this topic again or not(*), but I tend to agree with Eric that it's good to have the API open. If you are designing a theme for Read the Docs, you won't be able to query anything via the API. Let's say that you want to create your custom version menu flyout:

/api/v3/projects/pip/versions/?active=true

won't be available. What we would do in these cases?

(*) is all about communication with users?

There are some endpoints that need to be unauthed -- footer api, ads api, etc. But I'm 👎 on a public endpoints like /api/v3/projects listing. @davidfischer sums up the problems nicely.

I think requiring auth is pretty accepted API design at this point from a user perspective, and I don't feel that a public API provides enough value that we need to complicate our design for this use case. There is however a lot of value for us to keep things simple. If there is anything in particular that requires a public API, we should enumerate these though.

What we would do in these cases?

Use a API token :) Part of the intersphinx token work was to eventually pass in a generated token to the build process for this particular case, using our API from the build process.

ericholscher · 2018-11-05T17:51:18Z

docs/api/v3.rst

+* ``project``
+   * ``project:admin``: access to change admin settings (similar to Admin tab)
+   * ``project:read``: access to read all the information about the projects
+   * ``project:write``: access to anything related to perform write actions (triggering a Build for this project, activate/deactivate versions, etc)


How is this different than admin?

I'm thinking on a scope that allows you to trigger a build, but not to change the name of the project. From my point of view, admin will allow you to change anything under the "Admin" tab of your project dashboard.

ericholscher · 2018-11-05T17:53:15Z

docs/api/v3.rst

+Project details
+++++++++++++++
+
+.. http:get::  /api/v3/project/(string:slug)/


👍 to not exposing ID. slug is the public facing unique identifier in my mind. Though it is possible that it could change over time (by an admin), but I think that's an edge case.

humitos · 2018-11-06T12:34:49Z

I think it could use a design document explaining some of the design decisions that went into it. I also think it would be useful to have some explicit use cases that we care about, and want to support and aren't supporting. It feels like they are implied here, but having them be explicit would be very useful

I wrote a document that describes all the things that we have been talking, mentions the pros/cons and raise some questions. It's probably not 100% complete but we can add more things, discuss about the topics in this PR and we can write the decisions we take after that if we consider or leave them in the comments of PR.

For example, this API feels very focused on automating dashboard operations, but not doing anything with documentation page features, which is arguably ~90% of what our users want to be doing w/ RTD.

@eric could you expand on that?

What I understand from there is basically "having access to the documentation pages via the API". If I'm right on that, I'm basing that on "token scopes and serving any output file via the API" (as first approach at least) like /api/v3/projects/pip/docs/?filename=/objects.inv or similar. I know it's not the best, though, and I'm not sure if I can just use nginx serve file from that endpoint or not... but in any case, I think it's a good first start.

Let me know.

humitos · 2018-11-07T08:18:42Z

docs/design/apiv3.rst

+
+
+We do NOT want to cover
+++++++++++++++++++++++


Probably, we don't want to create users.

humitos · 2018-11-07T15:13:17Z

docs/api/v3.rst

+            "repo_type": "git",
+            "default_version": "stable",
+            "default_branch": "master",
+            "documentation_type": "sphinx_htmldir",


This field needs to be removed because it's deprecated.

stsewd

I left some comments, didn't read the previous discussions, sorry if I'm repeating something

stsewd · 2018-11-07T15:55:17Z

docs/api/v3.rst

+   We could extend the scopes to be per-project or per-user. I'm not
+   sure if this is possible yet, but we should consider it.
+
+   * per-project: the user could go to the Project Admin tab and create a Token for this specific project


I like this idea, similar to what we have for webhoooks (just access to /projects/)

stsewd · 2018-11-07T16:01:36Z

docs/api/v3.rst

+    :>json array links: Array with HEATEOAS_ links to retrieve related information
+
+    :statuscode 200: Success
+    :statuscode 404: There is no ``Project`` with this slug


We could add a top level section with all the status code and what they return (at the beginning it says we always return a json)

stsewd · 2018-11-07T16:03:06Z

docs/api/v3.rst

+Project import
++++++++++++++
+
+.. http:post::  /api/v3/project/import/


why not a post to projects/ endpoint (not import)?

stsewd · 2018-11-07T16:06:54Z

docs/api/v3.rst

+Project edit
++++++++++++
+
+.. http:patch::  /api/v3/project/(string:slug)/


I believe is more common to put <string:slug>

The syntax that I used comes from the .. http: directive and it uses the (string:slug) to show it nicer.

stsewd · 2018-11-07T16:18:15Z

docs/api/v3.rst

+
+   This endpoint may be under Project section
+
+.. http:post::  /api/v3/project/(string:slug)/builds/


or under /projects/versions/builds?

Without the slug?

If you missed the slug by mistake, I think it's not necessary to add /versions/ to the URL since "the project has builds" and each "build is tied to a version"

Yeah, I missed the slug

stsewd · 2018-11-07T16:19:33Z

docs/api/v3.rst

+                "API_HOST": "readthedocs.org",
+                "MEDIA_URL": "https://media.readthedocs.org",
+                "PRODUCTION_DOMAIN": "readthedocs.io",
+                "READTHEDOCS": true


Those are the only fields with uppercase keys

Yes, these come from the template context and I'm keeping compatibility/nomenclature from there.

stsewd · 2018-11-07T16:22:20Z

docs/api/v3.rst

+    :query string theme: Theme used in the documentation
+    :query string docroot: Docroot used to generate external links
+    :query string source_suffix: Source suffix used to generate external links
+    :query string format: Output format (``jsonp``, ``json``)


I guess jsonp is a workaround for CORS on custom domains? I think we want a better way to handle that.

stsewd · 2018-11-07T16:30:03Z

docs/design/apiv3.rst

+
+* Rate limit
+* ``Request-ID`` header
+* `JSON minified by default`_ (maybe with ``?pretty=true``)


I guess users can just use an API client that does the job to prettified the response

stsewd · 2018-11-07T16:31:04Z

docs/design/apiv3.rst

+* Rate limit
+* ``Request-ID`` header
+* `JSON minified by default`_ (maybe with ``?pretty=true``)
+* `JSON schema and validation`_ with docs_


Wonder if there is a sphinx extension to generate docs from the schema

stsewd · 2018-11-07T16:31:37Z

docs/design/apiv3.rst

+
+
+.. _JSON minified by default: https://geemus.gitbooks.io/http-api-design/content/en/responses/keep-json-minified-in-all-responses.html
+.. _JSON schema and validation: https://geemus.gitbooks.io/http-api-design/content/en/responses/keep-json-minified-in-all-responses.html


Looks like the wrong link p:

marcelstoer · 2019-02-05T13:09:03Z

docs/api/v3.rst

+
+    *See Build details*
+
+    :statuscode 201: Created sucessfully


Very much looking forward to this new API, thanks!

I suggest this returns 202 rather than 201 as "The request has been accepted for processing, but the processing has not been completed." (assuming the API is non-blocking of course)

codecov · 2019-02-19T15:34:41Z

Codecov Report

Merging #4863 into master will increase coverage by 0.05%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4863      +/-   ##
==========================================
+ Coverage   76.41%   76.47%   +0.05%     
==========================================
  Files         158      158              
  Lines        9990     9993       +3     
  Branches     1262     1262              
==========================================
+ Hits         7634     7642       +8     
+ Misses       2016     2009       -7     
- Partials      340      342       +2

Impacted Files	Coverage Δ
readthedocs/core/utils/__init__.py	`74.73% <0%> (-0.27%)`	⬇️
readthedocs/projects/views/private.py	`80.1% <0%> (ø)`	⬆️
readthedocs/doc_builder/python_environments.py	`82.97% <0%> (ø)`	⬆️
readthedocs/projects/models.py	`85.53% <0%> (+0.03%)`	⬆️
readthedocs/builds/models.py	`75.8% <0%> (+0.19%)`	⬆️
readthedocs/builds/views.py	`96% <0%> (+10.28%)`	⬆️

codecov · 2019-02-19T15:34:42Z

Codecov Report

❗ No coverage uploaded for pull request base (master@aae11fe). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #4863   +/-   ##
=========================================
  Coverage          ?   76.41%           
=========================================
  Files             ?      158           
  Lines             ?     9990           
  Branches          ?     1262           
=========================================
  Hits              ?     7634           
  Misses            ?     2016           
  Partials          ?      340

humitos · 2019-02-26T09:27:58Z

docs-lint is failing because,

./api/v3.rst:299: (INFO/1) No directive entry for "http:patch" in module "docutils.parsers.rst.languages.en".

but even installing sphinxcontrib-httpdomain doesn't fix it.

ericholscher · 2019-02-26T16:48:32Z

Looks like we can ignore it:

 --ignore-directives directives
                        comma-separated list of directives to ignore

stsewd · 2019-02-26T16:52:55Z

We have a configuration file for this https://github.com/rtfd/readthedocs.org/blob/master/docs/.rstcheck.cfg

ericholscher

👍 on shipping this, and iterating.

We do need to move the api/v3.rst file into a design directory though, since it isn't implemented.

stsewd

Isn't clear for me why we have the docs for the api and the design docs in the same PR, but the schema looks good.

docs/api/v3.rst

stsewd · 2019-02-26T18:23:50Z

docs/api/v3.rst

+
+
+Build command details
+++++++++++++++++++++


Not sure if we really need this one, I can't imagine why will want to list only one command, this info should be retrieved in /api/v3/projects/(str:project_slug)/builds/(int:build_id)/commands/

humitos · 2019-02-28T10:43:24Z

We do need to move the api/v3.rst file into a design directory though, since it isn't implemented.

I'm not sure what to do here.

We have design/apiv3.rst where some problems and proposals for APIv3 are written and api/v3.rst shows the current implementation, where the decisions from the design document are written and documented for when we publish APIv3.

On api/v3.rst I added a warning note saying that this is not ready to use since it's under development.

@ericholscher should I remove design/apiv3.rst and move api/v3.rst under design/ folder?

ericholscher · 2019-02-28T11:51:01Z

Perhaps we just move the actual user-facing docs into another PR where we can iterate on them with the code, like what Santos did for the YAML config.

stsewd · 2019-02-28T15:08:50Z

Don't we have a way of hide the document? Like make it draft

stsewd · 2019-02-28T15:16:28Z

Looks like we can use this http://www.sphinx-doc.org/en/master/usage/configuration.html#confval-exclude_patterns

stsewd · 2019-02-28T15:18:53Z

Perhaps we just move the actual user-facing docs into another PR where we can iterate on them with the code, like what Santos did for the YAML config.

BTW, that was kind of hard to maintain at the end :/, I would have preferred to put/update little chunks with the proper code.

humitos · 2019-03-05T19:34:57Z

docs/api/v3.rst

+    .. sourcecode:: json
+
+        {
+            "version": "latest",


I'm doubting about this now.

Instead of sending an argument here, we could do:

/api/v3/projects/(string:project_slug)/builds/ for triggering the default version configured (usually latest)

/api/v3/projects/(string:project_slug)/versions/(string:version_slug)/builds/ to trigger the specific version

This approach seems more clear and follow the pattern we are using, I guess.

Build is a weird one, the global pk communicates that this is a top-level object, but it effectively is not in any application. I see a use case for the first endpoint in the case of querying a build, but in the case of triggering a build, i'd definitely vote for the second option. In the case of querying a build, we don't necessarily care what the version is, but it might still help to be explicit about that relationship in our API modeling -- at least just for consistency?

Let's move this to a new issue though, the conversation here has dragged this PR out.

In the case of querying a build, we don't necessarily care what the version is, but it might still help to be explicit about that relationship in our API modeling -- at least just for consistency?

This was where I was doubting also. I like consistency but maybe that purity complicates the usage from a user perspective. Although, if we provide good links on the responses with the related object this shouldn't be a problem.

That's true too, yeah. Maybe also worth considering, long term, we might want to change the Build model pk to a compound key, in which case these two would be different builds:

/api/v3/projects/foo/versions/latest/build/1

/api/v3/projects/foo/versions/bar/build/1

The fact that we expose build pk as a global pk has always been sort of odd.

I like this idea.

Do we need to "modify the Build model pk"? Can't we make this an API identifier only for now?

1 would be the first (sorted by date) Build for this version

3 would be the third one

-1 would be the last one

Internally, we just sort the queryset (by date or -date) and pick the N element.

What do you think?

humitos · 2019-03-11T14:35:40Z

I removed the APIv3 user documentation from the toctree so it's not linked anywhere, but it's still available (with a warning message saying that's it not implemented yet) by hitting it the right URL manually.

I think we can merge it as is (this PR is insane with the amount of comments), and update the missing pieces/updates on #5356 (most of this PR is based on api/v3.rst document actually).

agjohnson · 2019-03-12T07:26:52Z

This has enough reviews to merge, feel free to merge if you think it's ready and we're tracking unanswered questions in another PR or issue.

jaraco · 2021-02-01T00:21:56Z

@humitos Thanks for the work on this. Were you aware there was a bounty for the feature to enable project creation by API? Please feel free to claim the bounty.

humitos · 2021-02-01T12:37:19Z

@jaraco nice! (I wasn't aware) I was paid by Read the Docs itself to do this work, so it may make more sense to donate this back to them 😄

On the other hand, I would appreciate a lot if you have some feedback to share with us about APIv3 and if it's has been being useful to achieve your needs.

jaraco · 2021-03-05T15:18:08Z

it may make more sense to donate this back to them 😄

The way BountySource works, an individual needs to claim the bounty, at which point I'll approve. I don't have the ability to withdraw a bounty and thereby contribute it elsewhere. I would very much like for you or an RTD owner to claim the bounty. Otherwise, it will just linger there indefinitely. Would you be willing to take the lead in finding someone to claim it?

On the other hand, I would appreciate a lot if you have some feedback to share with us about APIv3 and if it's has been being useful to achieve your needs.

It has been useful. It's been useful in updating the default branch, a routine I've used to facilitate the renaming of default branch across 100+ repos without having to click through a web UI. I still have plans to utilize the project creation API to automatically enroll projects mechanically, but I haven't actually implemented that yet.

humitos added the PR: work in progress Pull request is not ready for full review label Nov 5, 2018

humitos commented Nov 5, 2018

View reviewed changes

humitos added the Needed: design decision A core team decision is required label Nov 5, 2018

humitos commented Nov 5, 2018

View reviewed changes

stsewd reviewed Nov 5, 2018

View reviewed changes

humitos commented Nov 5, 2018

View reviewed changes

davidfischer reviewed Nov 5, 2018

View reviewed changes

ericholscher reviewed Nov 5, 2018

View reviewed changes

humitos commented Nov 7, 2018

View reviewed changes

docs/design/apiv3.rst

We do NOT want to cover

+++++++++++++++++++++++

Copy link

Member Author

humitos Nov 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, we don't want to create users.

humitos commented Nov 7, 2018

View reviewed changes

stsewd reviewed Nov 7, 2018

View reviewed changes

agjohnson added this to the 3.1 milestone Nov 13, 2018

humitos mentioned this pull request Dec 3, 2018

restapi: decouple ProjectViewSet from the pk #4192

Closed

agjohnson added the Feature New feature label Jan 25, 2019

humitos mentioned this pull request Feb 5, 2019

Trigger build through API #5231

Closed

marcelstoer reviewed Feb 5, 2019

View reviewed changes

humitos mentioned this pull request Feb 18, 2019

Design APIv3 #4286

Closed

humitos removed the Needed: design decision A core team decision is required label Feb 19, 2019

readthedocs deleted a comment from codecov bot Feb 19, 2019

humitos added 3 commits February 19, 2019 20:56

Initial idea about APIv3

719799e

Endpoint in singular

2bcff8b

More endpoints

3ace45b

humitos added 2 commits February 19, 2019 22:18

Link APIv3

208562c

Some fixes

0dff2d3

humitos requested a review from a team February 20, 2019 09:33

humitos added 2 commits February 20, 2019 12:26

More endpoints

b2e5ae5

Use query attribute to (not)show the config object on build details

2adee18

humitos removed the PR: work in progress Pull request is not ready for full review label Feb 20, 2019

Fix rstcheck

7146855

ericholscher approved these changes Feb 26, 2019

View reviewed changes

stsewd approved these changes Feb 26, 2019

View reviewed changes

stsewd mentioned this pull request Feb 27, 2019

Save relative paths from the config module #5364

Closed

humitos commented Mar 5, 2019

View reviewed changes

stsewd modified the milestones: 3.3, 3.4 Mar 6, 2019

Remove the user docs from the toctree

c5a4cea

humitos merged commit 874c920 into master Mar 12, 2019

delete-merged-branch bot deleted the humitos/api/v3 branch March 12, 2019 10:41

humitos mentioned this pull request Oct 1, 2019

APIv3 endpoint: allow to modify a Project once it's imported #5952

Merged


		This endpoint may be under Project section

		.. http:post:: /api/v3/project/(string:slug)/builds/



		.. _JSON minified by default: https://geemus.gitbooks.io/http-api-design/content/en/responses/keep-json-minified-in-all-responses.html
		.. _JSON schema and validation: https://geemus.gitbooks.io/http-api-design/content/en/responses/keep-json-minified-in-all-responses.html

Implementation of APIv3 #4863

Implementation of APIv3 #4863

Conversation

humitos commented Nov 5, 2018 • edited Loading

humitos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

humitos Nov 5, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidfischer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericholscher left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericholscher Nov 5, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

humitos commented Nov 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stsewd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

humitos commented Nov 5, 2018 •

edited

Loading

humitos Nov 5, 2018 •

edited

Loading

ericholscher Nov 5, 2018 •

edited

Loading