Skip to content

Conversation

@alinaliBQ
Copy link
Contributor

@alinaliBQ alinaliBQ commented Apr 10, 2025

Rationale for this change

  • Build Arrow Flight SQL ODBC DLL from flightsql and odbcabstraction component.
  • The ODBC driver will be available in Windows.

What changes are included in this PR?

  • Add handle allocation for environment and connection.
  • Add environment attribute setting

Are these changes tested?

Yes, on local Windows environment.

Are there any user-facing changes?

N/A

This PR addresses the following issues:

@github-actions
Copy link

⚠️ GitHub issue #46098 has been automatically assigned in GitHub to PR creator.

@alinaliBQ alinaliBQ changed the title GH-46098: [C++] Initial handle allocation in ODBC Layer GH-46098: [C++] environment & connection handle allocation in ODBC Layer Apr 10, 2025
@alinaliBQ
Copy link
Contributor Author

@aiguofer I have opened a draft PR for the ODBC layer.

@alinaliBQ alinaliBQ changed the title GH-46098: [C++] environment & connection handle allocation in ODBC Layer GH-46098: [C++] initial connection in ODBC layer May 2, 2025
@alinaliBQ
Copy link
Contributor Author

@zeroshade , @assignUser , @ianmcook Follow-up on the May-07 Arrow meeting, this is the draft PR that Rob and I have for creating the initial connection in the ODBC layer. It is based on top of @jbonofre 's PR#40939. Some changes in this PR are planned to be moved to PR#40939, such as the vcpkg.json update.

@jbonofre
Copy link
Member

jbonofre commented May 7, 2025

@alinaliBQ why not just using the original PR/branch ? Just to not loose changes/comments.

@alinaliBQ
Copy link
Contributor Author

alinaliBQ commented May 7, 2025

@alinaliBQ why not just using the original PR/branch ? Just to not loose changes/comments.

Yup you're right, for flightsql-odbc contribution, folks should use the #40939 PR, and that PR has higher priority for review than #46099. After #40939 PR is merged, we can go back to look at this #46099 PR.

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels May 7, 2025
@alinaliBQ
Copy link
Contributor Author

Hello @lidavidm , this is the draft PR that is based on top of @jbonofre 's #40939. It's still a draft PR but just wanted to give you a heads up

@alinaliBQ
Copy link
Contributor Author

alinaliBQ commented May 16, 2025

Hi @lidavidm @kou and other interested maintainers, please let me know if you prefer the Arrow Flight SQL ODBC contribution PRs like this one to be split into smaller PRs. It would be nice to know in advance so my team can prepare for the split, thank you!

@kou
Copy link
Member

kou commented May 17, 2025

Yes. I prefer smaller PRs to larger PRs because smaller PRs are easier to review than larger PRs.

(BTW, I'm not sure whether this PR is a smaller PR or not...)

@lidavidm
Copy link
Member

Yes, smaller PRs would be preferable. (To be frank, I have yet to actually review the first PR in full, but I'd rather get the paperwork done and get it merged ASAP, then we can go over it in detail afterwards.)

@alinaliBQ
Copy link
Contributor Author

@kou @lidavidm Ok sounds good! I will rebase Bit-Quill:apache-odbc on top of apache:main branch after #40939 is merged, so the size of this PR will be smaller compared to what it shows right now. Then I plan to open new smaller PRs to split this PR.

@alinaliBQ alinaliBQ changed the title GH-46098: [C++] initial connection in ODBC layer GH-46098: [C++] Arrow Flight SQL ODBC layer May 22, 2025
@alinaliBQ alinaliBQ changed the title GH-46098: [C++] Arrow Flight SQL ODBC layer GH-30622: [C++] Arrow Flight SQL ODBC layer May 23, 2025
@github-actions
Copy link

⚠️ GitHub issue #30622 has been automatically assigned in GitHub to PR creator.

@alinaliBQ
Copy link
Contributor Author

Hi David @lidavidm and Kou @kou, there's been a change regarding my last comment, we (Rob @rscales & I) will still work on splitting the PR into smaller ones and look into the flightsql-odbc comments from David on #40939 (and any other comments for Arrow ODBC Flight SQL), but that will be after we finish developing the ODBC layer to have the full functionality.
This PR#40939 will stay in draft state until we finish developing the ODBC driver, but any changes in this PR are ready for community to review.

TL;DR, we would like to finish the ODBC layer development first, and we will begin addressing the community comments after that.

Please let me know if you have any questions. :)

@lidavidm
Copy link
Member

OK - sounds good, please ping me when that is ready for review

@alinaliBQ alinaliBQ force-pushed the apache-odbc branch 2 times, most recently from cd83121 to 7edb145 Compare September 29, 2025 19:36
kou pushed a commit that referenced this pull request Oct 1, 2025
### Rationale for this change
Add initial framework for building an ODBC driver dll

### What changes are included in this PR?
- initial Flight SQL ODBC framework

### Are these changes tested?
- Yes, on local Windows environment

### Are there any user-facing changes?
- None
* PR is extracted from PR #46099
* GitHub Issue: #47516

Authored-by: Alina (Xi) Li <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
kou pushed a commit that referenced this pull request Oct 3, 2025
### Rationale for this change
Need to call function to register Arrow compute calls which is needed due to #25025

### What changes are included in this PR?
- Add calls to `arrow::compute::Initialize`
- Remove `RUN_ALL_TESTS` that wasn't needed; it is no longer needed after fix of GH-47434.
### Are these changes tested?
- `arrow-odbc-spi-impl-test` pass locally.
### Are there any user-facing changes?
No

PR is extracted from PR #46099
* GitHub Issue: #47679

Authored-by: Alina (Xi) Li <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
@alinaliBQ
Copy link
Contributor Author

Hi @lidavidm @kou, I got a question for you guys. 😃 I am considering opening a series of draft PRs that contain ODBC API implementation to get initial feedback and code review comments earlier. Currently we have around 40 ODBC APIs and each PR generally will have 1-2 APIs where it makes sense, so it will be many PRs raised. These ODBC API PRs still need to be merged consecutively to get the tests passing so that's why we are making them drafts. This approach helps us to get the ODBC driver available to the community sooner.

Are you folks good with this approach and okay with reviewing the ODBC API draft PRs? Please let me know your thoughts.

@kou
Copy link
Member

kou commented Oct 4, 2025

Yes. Let's use the approach.

@alinaliBQ
Copy link
Contributor Author

Sounds good. When the draft PRs are ready for a review, the team will tag folks as before

@alinaliBQ alinaliBQ force-pushed the apache-odbc branch 2 times, most recently from eedda11 to d9bb54c Compare October 8, 2025 22:13
@justing-bq justing-bq force-pushed the apache-odbc branch 2 times, most recently from b32f3e6 to 17f01d4 Compare October 10, 2025 16:49
@alinaliBQ
Copy link
Contributor Author

Thanks both for your feedback. I will be pausing the ODBC draft PR creation and our team will prioritize on the existing comments. I was excited from creating the PRs and got a little carried away. 😄
List of PRs created this week:

zanmato1984 pushed a commit to zanmato1984/arrow that referenced this pull request Oct 15, 2025
…e#47517)

### Rationale for this change
Add initial framework for building an ODBC driver dll

### What changes are included in this PR?
- initial Flight SQL ODBC framework

### Are these changes tested?
- Yes, on local Windows environment

### Are there any user-facing changes?
- None
* PR is extracted from PR apache#46099
* GitHub Issue: apache#47516

Authored-by: Alina (Xi) Li <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this pull request Oct 15, 2025
)

### Rationale for this change
Need to call function to register Arrow compute calls which is needed due to apache#25025

### What changes are included in this PR?
- Add calls to `arrow::compute::Initialize`
- Remove `RUN_ALL_TESTS` that wasn't needed; it is no longer needed after fix of apacheGH-47434.
### Are these changes tested?
- `arrow-odbc-spi-impl-test` pass locally.
### Are there any user-facing changes?
No

PR is extracted from PR apache#46099
* GitHub Issue: apache#47679

Authored-by: Alina (Xi) Li <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
@justing-bq justing-bq force-pushed the apache-odbc branch 2 times, most recently from 89fb490 to 6d02c36 Compare October 23, 2025 23:00
@alinaliBQ
Copy link
Contributor Author

Hi @lidavidm @kou Thanks for reviewing the PRs! Once we address the ODBC test review comments that are universal to the driver, do you think it is okay for us to open more new ODBC API draft PRs? 🙂
If yes, we will be creating about 20 more new draft PRs in total, and we would likely be raising several draft PRs a day.

Each day before we raise new PRs, we will first try to check to see if the review comments apply to rest of the driver, and the team will prioritize the comments that are universal to the driver. But if the review comment is only applicable for one draft PR, we will plan to work on it later if that's alright.

kou pushed a commit that referenced this pull request Oct 29, 2025
### Rationale for this change
 Enable ODBC Build In C++ Workflows to make sure ODBC can build in CI.

### What changes are included in this PR?
- Make environment variable `ARROW_FLIGHT_SQL_ODBC:ON` recognized in C++ build. 
- Register ODBC driver dll

### Are these changes tested?
Tested in CI
### Are there any user-facing changes?

No

Extracted from PR #46099
* GitHub Issue: #46004

Authored-by: Alina (Xi) Li <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
Co-authored-by: alinalibq <[email protected]>
Co-authored-by: rscales <[email protected]>
Co-authored-by: justing-bq <[email protected]>
Co-authored-by: vic-tsang <[email protected]>

ODBC commits squashed into 1 to make rebasing to main easier.

 Add initial framework for odbc dll

- Add ARROW_FLIGHT_SQL_ODBC option. If we set `ARROW_FLIGHT_SQL_ODBC=ON`, the flightsql odbc folder will be built
- Add odbc api layer for entry_point.cc
- builds odbc dll file, with ODBC APIs exported in odbc.def

Address James' comments

Fix `odbcabstraction` build errors and partially fix `flightsql_odbc` errors

Fix boost-variant not found error
- Adding dependencies from odbc/vcpkg.json to cpp/vcpkg.json

- Fix whereami.cc and .h dependency; ported lates code

Update whereami.cc

- use `long` instead of `int64`. Fixed namespace issues.

- PR CI fix: Add `parquet-testing` back

Partial build fix for `flight_sql` folder

- Replaced `namespace arrow` and `namespace odbcabstraction` with `using namespace ...`

- fix flight_sql_connection.cc

Fix `util::nullopt` to use `std::nullopt`

- fix std::optional

- fix BufferReader

- Fix GetSchema

- fix json_converter.cc

- partial fix configuration.h

-  partial fix get_info_cache.cc

- Fix winsock build error

- Comment out `flight_sql` files that cannot build
   - Comment out configuration and unit tests
   - Comment out get info cache and system trust store

Create initial odbc tests folder

Implement SQLAllocEnv

Fix cmake build

Implement SQLFreeEnv

Fix rest of build errors from `flightsql-odbc`

- Fix get info errors

- Fix for configuration window

  - added odbcinst library

- Fix system trust store

- unit test fixes

   - Add dependency of ARROW_COMPUTE. `arrow/compute/api.h` is used in
     `flight_sql`. Adding `ARROW_COMPUTE=ON` during build fixed run time unit
     tests failures.

Implement SQLAllocConnect and SQLFreeConnect

Fix build issue from static flight sql driver

Lint and code style fixes

Re-add deleted submodule parquet-testing

clang-format lint fix

cpplint lint fix

Exclude whereami in rat exclude list

C++/CLI lint fix

Update parquet-testing to match commit from `main`

Address Kou's comments

ODBC directory lint fixes

Catching the lint fixes outside of `flightsql-odbc` code

Fix build warnings that get treated as error

Implement SQLSetEnvAttr and SQLGetEnvAttr

Implement use of ExecuteWithDiagnostics

Doxygen Error Fixes and Address comments from Kou and James

Address comments from Kou

- Updates License.txt
- Update cmake toolchain
- Move whereami to `vendored`
- Use string_view instead of NOLINT std::string

Remove `whereami.cc` from arrow util build

We are building whereami.cc as part of odbc

Fix include headers to replace <> with ""

Address comments from James

Implement SQLGetDiagField

SQLDriverConnect, SQLConnect and SQLDisconnect

Implement stubs for SQLGetInfo, SQLGetDiagField and SQLGetDiagRec

Separate RegisterDsn and UnregisterDsn from windows build

Update code to save driver value from connection string

Add ReadMes for ODBC and tests

Fix test issues with string_view

Address code reviews

Update entry_points.cc to fix build issue

Remove Dremio references

Use emplace properly

Address comment from Rob and add SQLDisconnect test case

Add odbc.def and cmd file to rat_exclude

Nit - remove duplicate lines

Accidentally committed the change during git rebase

Nit - remove usage of nullptr

DSN window implementation

Add licenses to `.cmd` and `.def` files

Implement SQLGetDiagRec

Build ODBC in Windows Workflow

Tests are skipped for now.

Updates for SQLGetDiagFieldW

Enable Driver Logging

Add todo to update logging system later

Add logs

Implement ODBC API with debug logging

Enable mock test

* Add todo for noauth validation

* mock server with token auth

Add tests

* run same test with both modes

* Enable ODBC tests in workflow

* Switch current test cases to use FlightSQLODBCTestBase

So the tests can be skipped when `TEST_CONNECT_STR` is not set.

* Change tests to run on both mock and remote modes

Wrap usage of TEST_CONNECT_STR where possible

* Rename test fixtures and make connection string functions virtual

* Fix lint issue

* Attempt to enable ODBC build on Windows platforms

* Attempt to fix clang64 and MinGW errors

* Attempt to register ODBC

* Address James' comments

Use constant string for token

* use ServerMiddleware to validate token

Fix connection issues to DBT Labs

PopulateCallOptions before making a connection

Fix dsn window bug with advance properties

Fix seg fault issue from empty string

Implement SQLAllocStmt

Follow-up DBT Labs connection fix

Implement SQLGetDiag Rec and Field for statement

Unicode support for DSN ODBC APIs

* Let compiler append `W` to ODBC APIs where applicable.

Initialize Kernel functions

As per changes from apache#46261, we need to initialize Kernel library explicitly to get the functions registered

Implement SQLSetConnectAttr and SQLGetConnectAttr

ODBC Test Segfault Fix

* Move `connection_test.cc` to last in `CMakeLists.txt`, this resolves the seg fault. The seg fault was not reproducible on my local environment, even when I use the msys2 shell.
* Rename test executable to flight_sql_odbc_test

SQLExecDirect implementation

* SQLGetStmtAttr stub implementation

* Fix missing break statement in SQLGetDiagRec

* Run ShutdownProtobufLibrary as part of test environment

* Add comment for apacheGH-46889

* Pass call options to executed prepared statement

`call_options_` contains the authentication token which is needed

Implement SQLGetInfo

SQLGetInfo Workflow Fixes

SQLFetch & SQLGetData Implementation

SQLGetStmtAttr stub implementation

Stub call for SQLGetStmtAttr

Enable statement handle allocation

Implement SQLExecDirect - test hang issue

Update odbc_api.cc

Run `ShutdownProtobufLibrary` after all ODBC tests

Resolves test hanging problem

Fix missing break statement in SQLGetDiagRec

Add tests

Lint fixes

Run ShutdownProtobufLibrary as part of test environment

Add debug messages

Draft code

Remove drafts

Add comment

Add comment for apacheGH-46889

Update statement test headers

Pass call options to executed prepared statement

`call_options_` contains the authentication token which is needed

SQLFetch & SQLGetData initial impl

EVERYTHING before this commit is for SQLExecDirect

SQLFetch and SQLGetData TestSQLExecDirectSimpleQuery test

Make GetData() return SQLRETURN

`TestSQLExecDirectDataQuery` for remote and mock servers

Remove unneeded test case

Add more data types

* Add SQL_GUID in getCTypeForSQLType

* Add data types in query

* Add checks for columns 1 to 25

* Update statement_test.cc

Add varbinary test

* Implement SQLNumResultCols

Will leave tests for later

* Implement SQLRowCount

* Implement SQLMoreResults

* Work on date retrieval fix

The `TestSQLExecDirectDataQuery` checks for date is not working.

* Fix date retrieval error by importing fix from Paul

* imported fix from dremio/flightsql-odbc@d44d862, an Apache-Licensed repository

* Add more checks for get data

* Add more tests

* Continue work on float truncation test

* Disable float truncation test

* Address comments from James

- do not return errors for invalid rowCountPtr and columnCountPtr
- add static cast for columnCount
- Add tests with SQL_C_DEFAULT as target type
- Add test checks for indicator
- Fix SQL_NUMERIC, SQL_BIT, SQL_FLOAT target type

* Fix build issues in CI

* More fix for CI

* Add test for invalid buffer length

SQLPrepare & SQLExecute Implementation

* Add tests
* Address comments from Rob

Code Style fixes

* Move ifdefs outside of test cases

* use `stmt`

* Move logs to first line

* fix test for `SQL_DRIVER_HSTMT`

* Use camel case for tests

Implement SQLGetStmtAttr and SQLSetStmtAttr

SQLBindCol Implementation

* SQL_UNBIND implementation + tests

* Add tests for `SQL_ATTR_ROW_ARRAY_SIZE`

* Add check for `SQL_ATTR_ROWS_FETCHED_PTR`

Enable TestCloseConnectionWithOpenStatement test case

SQLFetchScroll Implementation

* Add checks for SQL_ATTR_ROWS_FETCHED_PTR

* SQLFetchScroll is supposed to return the number of rows fetched using SQL_ATTR_ROWS_FETCHED_PTR

SQLExtendedFetch Implementation

* Implement rowCountPtr and rowStatusArray for SQLExtendedFetch

* Add tests for SQLExtendedFetch

* `SQLExtendedFetch` doesn't return `SQL_SUCCESS_WITH_INFO` for error state 22002.

* Add tests for `SQL_ROWSET_SIZE`

* Address comments from James

apacheGH-46584 Iterate endpoint

* use suggestion from James for one-liner change to return `Status::OK` directly

* fix for iterating endpoints, it is based on JDBC's fix for the same bug

* save value of `FlightClientOptions`

* Use `TestFlightServer`

Add `arrow_flight_sql_shared` to enable usages for `TestFlightServer`

* Fix build errors from missing `gmock`

* Add checks for array data

Update flight_sql_stream_chunk_buffer_test.cc

Add `FlightStreamChunkBufferTest` unit test

Draft test with `FlightSQLODBCMockEndpointTestBase`

* Reference apacheGH-47117 and Clean up code

* Add driver and DSN to built-in properties to not pass driver/dsn as attributes to server

* Allow `=` in values inside connection strings, fixes connectivity problems

* Address comments from Rob

SQLColumns Implementation

* Initial impl for SQLColumns

* Add test for SQLColumns

* Add columns test with all supported column types

- mock server doesn't support schema

* Address comments from James

- Test different data type return value for SQLColumns
- Add todo comment for SQLDescribeCol tests

---------

Co-authored-by: rscales <[email protected]>

Fix bug of DSN not read properly

* Fix reading DSN bug

Use a simpler, more robust way to load the DSN

SQLColAttribute implementation

* SQLColAttribute initial impl

* Switch to use `GetAttributeSQLWCHAR` to be unicode-compatible

* Add SQLColAttribute tests and fix bugs in `flightsql-odbc` impl

* Fixed bug with incorrect column count. It was returning column count + 1.
* Fixed bug with incorrect numPrecRadix. It was returning SQL_NO_TOTAL (maps to -4) for non-numeric columns, but the correct value is 0.
* Fixed bug with unsigned column to return true for unsigned columns (non-numeric columns or unsigned numeric columns) and false for signed columns (numeric columns)
* Fixed bug with incorrect non-concise data type return value for date time type types. The correct return is SQL_DATETIME for all date time types

* Address comments from James

* use const for `ConvertToWString`
* add nullptr check in odbc_api.cc layer
* Add support for `SQL_COLUMN_LENGTH`, `SQL_COLUMN_PRECISION`, and `SQL_COLUMN_SCALE`. The driver will return the same values for ODBC2 and ODBC3.
* Add tests for ODBC v2 SQLColAttributes values
* I converted tests that can be run on remote servers where possible

* Address comment from Rob

Implement SQLTables

Use C++ 20 standard for ODBC

* Add support for building with C++20

* Upgrade spdlogs version as an attempt to fix build issues

* Fix issue of missing unique_ptr definition

Add tests for SQLMoreResults

Implement SQLNativeSql

Add diagnostic records

* remove `SQLErrors`, as the driver manager is supposed to map `SQLErrors` to `SQLGetDiagRec`

Implement tests for SQLNumResultCols

SQLCloseCursor Implementation

* fix error state to return the correct state 24000.

* add tests for SQLCloseCursor and SQLFreeStmt(SQL_CLOSE)

Add SQLColumns and SQLColAttribute tests with nullptr

* Modify SQLTables test to run on both mock and remote
* reorder error msg alphabetically

Implement tests for SQLRowCount

Fix merge conflicts

Adjust searchable return value

based on apacheGH-46920 fix

Fix `getCTypeForSQLType` return for interval types

- Fix `getCTypeForSQLType` function to return the correct `C type` for interval SQL data types. Previously the function was checking for interval `C type` and returning interval `SQL type`, which was the opposite of the intended functionality.

SQLGetTypeInfo implementation

* Add tests for SQLGetTypeInfo

* fix bug of createParams returning empty string instead of null
* fix bug of non-concise data type being returned for datetime/interval
* fix bug of longvarchar not converted to wlongvarchar when driver has unicode enabled

* Address comments from James and add checks

*  test SQL_ALL_TYPES in ODBC 2.x.
*  test SQL_TYPE_* in ODBC 2.x, the driver manager reports invalid data type error.
*  test SQL_* datetime in ODBC 3.x for backwards compatibility.
*  add check for valid/invalid data type.

Fix segfault issue from empty metadata

* Use empty map in bug fix

* Address comments from James

Implement SQLDescribeCol

Add SQLError Tests

* add test to free null handles. Without handle value initialization, segfault error was seen

Move `SQLGetDiagField` and `SQLGetDiagRec` tests to `errors_test.cc`

* Address comments from James

* Add ODBC Ver 2 tests

* update test name to indicate if error handling is from driver manager.
* add tests for warnings.

* fix lint errors.
* remove SQL_ATTR_APP_ROW_DESC that is not applicable to Environment Attribute.

Add tests for SQLGetFunctions

* Using `SQL_FUNC_EXISTS` macro fixed the issue of `api_exists` not read correctly

* Fix SQLGetTypeInfo naming for ODBC ver 2 tests

* Add more tests

Add unsupported API checks.
Add `TestSQLGetFunctionsODBCVer2`.
Add `TestSQLGetFunctionsCheckSingleAPI`.

* Update reset value

* Add SQLDescribeCol function check

Add additional SQLDescribeCol test cases

Add Descriptor support in SQLAllocHandle and SQLFreeHandle

* Descriptor allocation initial impl

* Add descriptor handle tests

* Add diagnostic error test for descriptor handle

* the error is from driver manager as our driver doesn't have descriptor-specific APIs implemented yet

* Add SQLGetDescField test from driver manager

* Add doxygen doc comment

Fix connection issues servers that require data in handshake

* revert back to passing empty string in handshake

Fix bug of system trust store not loaded properly

* Bug: system trust store has default value of "true" regardless of "encryption" value.
* But if encryption is set to false, system trust store cannot be used, so system trust store should be saved as false if user does not enable encryption.
* It may confuse the user if they see encryption is set to false but system trust store is set to true. In this case, the driver will not do system trust store verification. The DSN window should reflect this accurately.

CPack ODBC Windows msi installer

Add ARROW_FLIGHT_SQL_ODBC_INSTALLER environment variable to enable creation of Windows installer.

* Use CPack + WiX to create a `msi` installer.
* run command `cpack` under the build directory to generate the installer.

Register ODBC Driver on Windows in installer

* In `wxs`, cannot use `package`, need to use `fragment` instead

* Use component as feature will automatically be generated

* Set Patch version + add installer instructions

Use Arrow logo banner

* replaces default banner with Arrow logo banner

Indicate Driver Company and Version on Driver Manager

* use versioninfo.rc.in template

* use `1 VERSIONINFO` for it to work properly

* Set version variables

* Add `.rc` to gitignore

* Add `@` variables to rc template

Return nameLengthPtr value as length in characters

Allow spaces while parsing Table Type

mac changes

added changes to build in MacOS

Enable ODBC build on Windows Glib & Ruby workflows

* Fix `boost::get` issue on GLib workflow

* Fix GLib build issue with conflicts

Compile entry_points.cc before odbc_api.cc due to conflict from sql.h and flight/types.h.

Fix build warnings treated as error on Windows

SQLDescribeCol expects `bufCharLen` to be of type `SQLSMALLINT`, so build warnings occur when `bufCharLen` is set to `SQLINTEGER`.

Add more types for Array Conversion

Set column attribute SQL_DESC_TYPE_NAME

[Refactor] Remove reference of `Configuration` in odbc_connection.cc

Remove reference of `Configuration` in odbc_connection.cc which is in library `odbcabstraction`. The reasoning is that `Configuration` is in `arrow_odbc_spi_impl` and this is a library that `odbcabstraction` depends on. I moved the logic for reading DSN values into `odbc_api.cc` instead, since that is a place that will result in no conflicts. The ODBC driver now does not save the DSN value in the connection string if the DSN value is put after Driver value.

* Address comments from James

Simply logic to check for first key value pair only

Add TODO for non-UTC time zone support

* Currently, the driver only supports UTC time zone

Replace `spdlogs` with Arrow's Internal Logging

* Add logging README

* register kernel function conditionally

   * Resolves errors of kernel function already registered

Logging files are not supported since GLOG is not enabled on Windows in Arrow repo. We can work + test on log file support on macOS/Linux phase

Remove  `RUN_ALL_TESTS` as they are not needed

Previously, tests on Windows were not being run due to apache#47434. So we added  `RUN_ALL_TESTS`  as a workaround in early May, before apache#47434 was raised.
Now that apache#47434 is resolved, we can remove  `RUN_ALL_TESTS` as they are not needed.

Other changes:
* Renamed `arrow_odbc_spi_impl_test` to `odbc_spi_impl_test` so the test executable will be  `arrow_odbc_spi_impl_test.exe` instead of  `arrow_arrow_odbc_spi_impl_test.exe`

Fix Function & Variable Naming

Fix member variable naming

Address code review comments for Arrow#40939

* Reword comment on null check

Addresses comment apache#40939 (comment)

* Use `std::memcpy` instead of `memcpy`

Addresses comment apache#40939 (comment)
* add include cstring

* Remove warpdrive mentions

* Fix lint

* Address Justin's comments

Combine utils and define utils namespace

Use arrow::flight::sql::odbc namespace

Update namespaces driver::flight_sql and driver::odbcabstraction to arrow::flight::sql::odbc.

Fix seg fault from register log

PR to keep apache-odbc in sync with arrow main branch.

I found that, if the driver registers logs before compute kernel registration, segfault somehow occurs at when register kernel function is being used. I am not sure the root cause, but registering the logs after compute kernel registration seems to resolve the issue.

Refactor odbcabstraction and odbc_impl

flight_sql is renamed to odbc_impl. The files inside odbcabstraction have been moved to odbc_impl and odbcabstraction is removed.

Replace boost::variant with std::variant

* Remove `boost-optional` and `boost-variant` dependencies from vcpkg.json

* Replace boost::variant with std::variant

Fix EXPECT_EQ() to have expected on the left and actual on the right

Code style naming convention

Remove underscores in test names

Sync arrow_flight_testing changes from Arrow::main branch

Fix test failure due to typo

Fix `FlightSQLODBCRemoteTestBase.TestSQLExecDirectDataQueryDefaultType` test failure due to typo
The precision should be 38 in the original code. I think it got changed to 0 by accident somewhere.

Disable tests with `DISABLED_`

* Enable `TestSQLGetInfoDriverHdesc`

We missed this test before, enabling the test now

* Disable tests with `DISABLED_`

Fix test assertions

Define Test Fixtures

* Connect/disconnect in SetUp/TearDown
* Define Test Fixtures

Resolve ODBC CI Build Errors

* Fix `ConfigDSNW` not found error

* Fix `TestFreeNullHandles` test

`ConnectionTest` sets up the handles, so the `this->...` handles will not be null at beginning of test.

Define New Base Test Fixtures

Skip tests without connecting/disconnecting

Address general test code review comments

* Use RAII helper for allocating and freeing env/conn handles

Avoids duplicated code

Move test helper functions to anonymous namespace

Add `[[nodiscard]]` for ODBC APIs

Addresses community comment:
https://github.com/apache/arrow/pull/47763/files#r2450014186

Remove `using List=` in test suite definitions

Use static_cast for `SQLWCHAR` in type info test

Use static_cast for `SQLWCHAR` in tables test

* use mutable arrays for places where characters cannot be const

Use static_cast for `SQLWCHAR` in columns test

Update comment

Use static_cast for `SQLWCHAR` in SQLGetInfo test

* Address Justin's comments

* Add fix for skipping the allocation of handles
@alinaliBQ
Copy link
Contributor Author

I hope this makes it easier for reviewing: If folks see any general comments on our draft PRs, feel free to mention that it is a general comment, and our team will work on these on all applicable PRs. Please don't feel the need to post the general comments on all the applicable PRs.
For in-progress PRs that don't have all known general comments addressed, we will label them with WIP.
Thanks for helping to review/merge our PRs!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants