ARROW-15452: [FlightRPC][Java] JDBC driver for Flight SQL #12830
Conversation
…tCerts` to TLS connection Tests.
…nverted to SqlTypes.
* Address all ratification comments with small fixes * Small fixes * Revert small changes * Add more exception messages * Remove ExceptionTemplateThrower * Change UnsupportedOperationException to SQLException * Address all ratification comments with small fixes * Fix failing tests * Address to James' review * Removed some suppressions by creating newChildAllocators * Fix CONNECTION_STRING_EXPECTED * nit on CONNECTION_STRING_EXPECTED * Address more review comments * nit on holder comment * Remove FindBugs annotations * Fix rebase issues * Downgraded JUnit version back to what is defined in java/pom.xml * Remove JUnit 5 usage * Fix import order * Format rebased commit * nit: add final
…d a Builder `typeName`.
[JAVA] [JDBC] Jdbc tls connection with disable certification verification
This PR adds the feature to include the filename of a fragment if it is a FileFragment as a scalar value and includes the augmented fields in the dataset schema Closes apache#12560 from sanjibansg/filename_retrieval Authored-by: Sanjiban Sengupta <[email protected]> Signed-off-by: Weston Pace <[email protected]>
https://nvd.nist.gov/vuln/detail/CVE-2018-25032 zlib before 1.2.12 may cause memory corruption Closes apache#12760 from cyb70289/16078-zlib Authored-by: Yibo Cai <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
…tObject-getString' into change-interval-behaviour-for-getObject-getString
Add support for ";" as parameter divider
Implement toString method for JdbcArray class
Fix struct type on jdbc
* Add toString to Time obj in Time#toString * Improve Time toString * Fix maven plugins * Revert "Update java/flight/flight-jdbc-driver/src/test/java/org/apache/arrow/driver/jdbc/accessor/impl/calendar/ArrowFlightJdbcTimeStampVectorAccessorTest.java" This reverts commit 00808c0. * Revert "Merge pull request #29 from rafael-telles/Timestamp_fix" This reverts commit 7924e7b, reversing changes made to f6ac593. * Fix DateTime for negative epoch * Remove unwanted change * Fix negative timestamp shift * Fix coverage * Refator DateTimeUtilsTest
|
@jduo do you still plan to rebase things? Also, would you like the base branch to be updated since the proposal was merged? |
…ormat Change the behavior to return the correct format for day and year intervals.
Fix query exception message
…etadata-type-name [Java] [JDBC] Flight jdbc driver create metadata type name.
|
Hi there, |
|
Hi, I've tried this code and found it supports auth token only when set the server auth handler as NoOpAuthHandler and add an auth header in middleware to pass the token at the same time, right? |
|
Looks like everyone's filed ICLAs. I'll get the paperwork filed ASAP |
| public List<FlightStream> getStreams(final FlightInfo flightInfo) { | ||
| return flightInfo.getEndpoints().stream() | ||
| .map(FlightEndpoint::getTicket) | ||
| .map(ticket -> sqlClient.getStream(ticket, getOptions())) |
There was a problem hiding this comment.
I think this behavior is incorrect, and it is causing problems for DataFusion's implementation. It looks like this code re-uses the existing sqlClient, ignoring the value of flightInfo.enpoints[x].locations[y].uri (which may be different than the original URI the sqlClient connected to, in the case of DataFusion, the sqlClient would be connected to the scheduler at one URI, and the data would be held on one or more executor nodes at different URIs).
I can attempt a PR if you agree with my hypothesis. I'm new to all this, so I could be way off base. And thank you for putting in the massive amount of work necessary to make this BTW!
There was a problem hiding this comment.
You are right, we actually merged a clarification for this behavior a little while ago. #12636
Note we're still finalizing the IP clearance of this PR (but I assume it will go through one way or another)
There was a problem hiding this comment.
Glad to hear it's been fixed. Is there a current, long-running repo & branch we can track until this gets merged into arrow proper?
There was a problem hiding this comment.
This is the repo/branch being used for the IP clearance so I would say you're already looking at it!
My plan is once the clearance goes through, this will get merged into a branch on apache/arrow (the PR merge target here, flight-sql-jdbc), then we'll merge master into the branch, then finally do a PR into master (and review the code and such).
There was a problem hiding this comment.
Oh, I think I understand now:
merged a clarification for this behavior
The behavior has been clarified, but the code to which I linked has not been corrected according to the now clarified expected behavior?
If that's the case, I'll work on a fix and PR this branch.
There was a problem hiding this comment.
Yes, you got it. We clarified the spec, but haven't necessarily updated everything else
There was a problem hiding this comment.
This works, but I'm interested in feedback about how to do it the correct way: rafael-telles#42
I'm also happy to help others get set up to run with a similar test environment as me if that helps.
There was a problem hiding this comment.
I see James followed up already. Thanks for taking a look!
Fix attributes as headers bug and token value parsing error
lidavidm
changed the title
19 participants
This implements a JDBC driver able to communicate to Flight SQL sources.
So far this covers:
Yet to be done: