Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Implement ExecuteQuery API for SQL support #2280

Merged
merged 18 commits into from
Jul 17, 2024
Merged

feat: Implement ExecuteQuery API for SQL support #2280

merged 18 commits into from
Jul 17, 2024

Commits on Jul 16, 2024

  1. feat: publish the Cloud Bigtable ExecuteQuery API 

    The ExecuteQuery API will allow users to query Bigtable using SQL

PiperOrigin-RevId: 650660213

Source-Link: googleapis/googleapis@f681f79

Source-Link: googleapis/googleapis-gen@3180845
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMzE4MDg0NTQ4NzEzNjc5NDk1MmI4ZjM2NWZlNmM2ODY4OTk5ZDljMCJ9
    @gcf-owl-bot @jackdingilian
    gcf-owl-bot[bot] authored and jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    0ea298c View commit details
    Browse the repository at this point in the history

  2. 🦉 Updates from OwlBot post-processor 

    See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
    @gcf-owl-bot @jackdingilian
    gcf-owl-bot[bot] authored and jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    f82b891 View commit details
    Browse the repository at this point in the history

  3. feat: publish ProtoRows Message 

    This is needed to parse ExecuteQuery responses

PiperOrigin-RevId: 651386373

Source-Link: googleapis/googleapis@a5be6fa

Source-Link: googleapis/googleapis-gen@d467ce8
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZDQ2N2NlODkzYTA0YzQxZTUwNDk4MzM0NmMyMTVkNDFmZDI2MzY1MCJ9
    @gcf-owl-bot @jackdingilian
    gcf-owl-bot[bot] authored and jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    3c129ba View commit details
    Browse the repository at this point in the history

  4. 🦉 Updates from OwlBot post-processor 

    See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
    @gcf-owl-bot @jackdingilian
    gcf-owl-bot[bot] authored and jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    3379328 View commit details
    Browse the repository at this point in the history

  5. Temporarily allow method level tests to pass for ExecuteQuery 

    I will re-add these once the client supports the appropriate behavior for readRows

Change-Id: I4b5b0861e93f0a8b97d51556c74bccfe0b1da521
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    b06e9d4 View commit details
    Browse the repository at this point in the history

  6. Copy SqlRowMerger from internal code 

    This moves things mostly as is. I will move everything to appropriate packages in a follow on (e.g. utils will move to internal). Wanted to keep the changes as minimal as possible

Renames ProtoRowsBatchMergingStateMachine to ProtoRowsMergingStateMachine and removes the obsolete ProtoRowsMergingStateMachine

This simplifies the internal code by removing support for the old api. It should not change any behavior for he ProtoRowsBatch API

Change-Id: I8e8267a9cfeb59e0f8455876d1bccdb7c3778b76
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    9cfe1a0 View commit details
    Browse the repository at this point in the history

  7. Implement main interfaces for interacting with ResultSets 

    This adds the main interfaces for interacting with data:
ResultSet, StructReader, and Struct. StructReader is used
by ResultSet, SqlRow, and Struct to provide a standard set
of accessors for the data.

I'm a little uncertain about the package structure right
now. I've put interfaces that users will interact with
directly in 'models' and everything else in 'internal'.
Internal will include both internal interfaces and the
implementations of the public interfaces. Otherwise I have
kept the package structure pretty flat but happy to group
the sql stuff together more.

This includes thorough documentation for the public interfaces.

Change-Id: I4535daffc596e9355da64b5655c58a68e5e7a95f
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    c732dd0 View commit details
    Browse the repository at this point in the history

  8. Implement proto to java conversion via AbstractProtoStructReader 

    This will be re-used for structs in the following commit. The current
design implements this at the Row/Struct level instead of ResultSet &
Struct so that result set can operate on rows of different types when we
support arrow. ResultSet will then be a light wrapper where each getType
call is passed through to the current row.

Change-Id: I21fb4cc0cd830110d5455581491c5b1861efbcb1
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    ed96cbf View commit details
    Browse the repository at this point in the history

  9. Add implementation for ProtoStruct 

    Change-Id: Ie9d3892fe92c779a6b81d457c31b534bdce20f38
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    ab676df View commit details
    Browse the repository at this point in the history

  10. Implement ResultSet 

    This also removes the metadata helpers from the ResultSet interface.
These helpers will live on ResultSetMetadata instead.

Change-Id: Iba60b42b4e62f2d72c5a21206edf716c0ad5ea73
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    cb96881 View commit details
    Browse the repository at this point in the history

  11. Implement ResultSetMetadata and use it everwhere we operate on metadata 

    Also fix AbstractProtoStructReaderTest which was using the wrong runner
and not executing any tests. Fixes a couple test bugs as well.

Change-Id: Ia59459f93dd38046e9ac22995191becaad6ea44e
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    e40c924 View commit details
    Browse the repository at this point in the history

  12. Create execute query callables and wire up api to data client 

    This uses the ExecuteQueryCallContext to pass the metadata back
to the user using a future. Note that we need to be very careful
here that errors always close this future. That is why watchdog
happens higher in the call chain that for other streaming RPCs

Also note i decided to unwrap executeExceptions so that when
possible the future throws the same exceptions as the row stream

Change-Id: Ie9dd523a980efecb2cff010a2ffd7ac52b5f8ac7
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    d364c1d View commit details
    Browse the repository at this point in the history

  13. Add statement API for ExecuteQuery requests 

    Change-Id: Id9113adb24419da18f6e43e47c0c73cddf4b622e
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    bbcd0f0 View commit details
    Browse the repository at this point in the history

  14. Implement java representation of Type protobuf 

    The admin types in admin.v2.models.Type will be migrated to
1) share the same implementation in common, while defining a
narrow interface like SqlType 2) be renamed to SchemaType.
We aren't going to block the Sql launch on this though. It
can happen after.

Change-Id: I9dea84fe9b7b2e0b46e8ece4cb666651183dc639
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    c405206 View commit details
    Browse the repository at this point in the history

  15. Remove unsupported query param types and add doc 

    Change-Id: I41a28069865df6aa688ac5bc96181aae9f18deb4
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    aefb16d View commit details
    Browse the repository at this point in the history

  16. Add clirr exclusion for new proto method. Fix dropped import 

    Change-Id: I0cf3bd45a5f2bb3e2ad980d4170049d69e701d7e
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    8e090fc View commit details
    Browse the repository at this point in the history

  17. Small doc update and remove obsolete TODOs 

    Change-Id: Ie08217d025df3ec3c97dc96268f1c4e1d20b33bf
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    0207e1d View commit details
    Browse the repository at this point in the history

  18. Minor documentation fix for executeQuery 

    Change-Id: I6073e41b5a5b74b91671002cb93dc42c4023f10f
    @jackdingilian
    jackdingilian committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    10e69e0 View commit details
    Browse the repository at this point in the history