Possible API alternative

[ggriffiniii]

First let me say thanks for generating all these API's, the rust ecosystem needs them and they provide a ton of value.

I have an idea and proof of concept for how to interact with google API's in rust that I think has a lot of benefits, and given that your libraries are the de facto way of using google API's in rust I wanted to run it by you in case you had any thoughts.

Most of the modern google API's provide a fields attribute on the request that controls which attributes you want populated in the response. You can see the google drive API for the fields attribute here, but most of the current API's have identical functionality.

Your currently generated API (along with API's in other languages) handle this by having a response struct that contains a bunch of optional fields. For example google_drive3::File currently has over 50 fields each of which is optional and some have nested sub-fields. The two big issues with this are that while it's type-safe the type itself does not represent what fields it effectively contains, and because the struct maintains every possible field it's larger in memory than all but the most extreme cases require.

As a user of the library I wished I could define my own struct that implemented serde::Deserialize and have the responses deserialized into my struct. This would allow me to express my data in a more rich way using the type system and use the powerful features of serde to deserialize into appropriate types like chrono::DateTime where appropriate. This is pretty straightforward, but now the fields attribute on the request had to be crafted to match the fields specified on my struct. To solve this I created a procedural macro that can automatically derive a field_selector. When the field_selector is provided in the request it will request all the fields in the struct. I'm pretty satisfied with the way it ended up and thought it might make for an interesting approach for the generated API's you provide.

My proof of concept can be found here:

• https://github.com/ggriffiniii/google_api3
  • That repository defines the FieldSelector trait along with a proc macro to derive it on structs. Any struct that implements FieldSelector can call T::field_selector() to generate a selector for the fields attribute.
• https://github.com/ggriffiniii/simple_drive_api
  • This repository has a barebones implementation of a google drive client that only lists files. It's intentionally stripped of a lot of functionality to highlight how FieldSelector can be used. The concept of how FieldSelector is used can be applied to the vast majority of modern google api's.
  • In particular https://github.com/ggriffiniii/simple_drive_api/blob/master/examples/list_files.rs has an example of how a user of such a library would operate. Note that it's doing 2 different file list operations and each is returning a different struct with different fields populated.

There are some tradeoff's here. Some users may not want to define their own structs. In most cases, I suspect, users are already converting from google_drive3::File into a more appropriate struct for their application, this would just allow them to deserialize directly into their struct. Additionally, the current behavior could overlap with the FieldSelector approach if needed. For example google3_drive::File could implement FieldSelector, and the type provided field_selector could only be used when the user doesn't provide an explicit fields attribute. You could also provide a couple methods with different names.

fn list() -> google3_drive::File and fn list_into<T>() -> T

where list() is implemented as list_into()

I think that pretty much sums up my thoughts. If you're interested in this approach and would like to work towards merging it into the generated API's somehow I would be happy to help. It's fine if you're not interested as well. I'm happy to continue using it just in my private projects, I just thought it might be an opportunity to improve the ecosystem as a whole.

[Byron]

First of all, sorry for the late reply!
Please know that I highly appreciate the work you have done, and love the idea of integrating the Google APIs more neatly with Rust and Serde, allowing people to be even more efficient when needed.

I have had a look at the demonstration for the proc-macro and its use and love it. Personally I believe it is indeed possible to provide the additional option similar to what you are suggesting, i.e. list() -> File and list_into() -> T, and remain fully backwards compatible.

If you do still have the time to take a look on how this could be integrated into this repository, please give it a go.

Also CC @TheBiggerGuy as he is a contributor who can possibly merge PRs when I can't.

[ggriffiniii]

I'll look into doing this when I get a moment. One thing I realized is that the builder pattern used will not make the name as intuitive.

The google drive file list api for example is currently looks something like:

let files = hub().files().list().doit();

The function that needs to have a generic return type is doit(), not list(). So rather than being list() vs list_into() it will need to be doit() vs doit_into(), which in my opinion doesn't read nearly as well. There are a number of options. Most are not very technical. I'll try and put together a change that adds a straightforward doit_into. Once we see what it looks like we can think if there are any better alternatives.

On an unrelated note. Is there a reason the generator is written in python/make rather than using rust and one of it's template systems? It's a pretty minor thing, but it would be great if the generator was a simple cargo build away rather than needing to pull in all those python dependencies (the makefile is currently broken because the download location for virtualenv appears to have changed).

[Byron]

For anyone reading this I may add that by now there is an entirely new generator which might work better for you:

https://github.com/google-apis-rs/generator