Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

ODBC: Use literals instead of parameters in Power BI data connector #652

Merged
merged 36 commits into from
Aug 3, 2020
Merged

ODBC: Use literals instead of parameters in Power BI data connector #652

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
427bd79
pbi connector
Jul 19, 2020
401a053
update names
Jul 19, 2020
11856cd
- add mez file
Jul 20, 2020
e693269
nit
Jul 20, 2020
4158974
nit
Jul 20, 2020
586793f
image resize
Jul 20, 2020
9cb48a4
resize image
Jul 20, 2020
c234b88
remove extra lines
Jul 20, 2020
0d0c050
nit
Jul 20, 2020
50d4ccf
update mez file
Jul 20, 2020
023fb67
rename connector
Jul 21, 2020
0b857c0
rename mez file
Jul 21, 2020
06ae486
- add basic & aws auth
Jul 23, 2020
b2ff222
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 23, 2020
3f6ff5d
add connector
Jul 23, 2020
ad73d91
update doc
Jul 23, 2020
91d5cbe
add ssl for auth
Jul 23, 2020
ddc725b
update docs for errors
Jul 23, 2020
b946351
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 23, 2020
2bd944c
- remove sqlgetinfo & sqlcapabilities values from connector
Jul 24, 2020
ed0afb9
adding EncryptedConnectionString
Jul 24, 2020
22dc963
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 28, 2020
82e9a72
add data type for columns
Jul 28, 2020
5022f1f
add alternate column size
Jul 28, 2020
f4c7068
reverting changes in catalog test
Jul 28, 2020
a76c382
nit
Jul 28, 2020
debe0c5
revert changes for column size validation
Jul 28, 2020
6add9da
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 28, 2020
a80eee8
address PR comments
Jul 29, 2020
e6490e4
Update SQL_GROUP_BY value for fixing query folding issue
Jul 29, 2020
e1032c3
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 29, 2020
9a36d47
removing AstVisitor for resolving query folding issue with data type …
Jul 30, 2020
8fa089b
fix parameter binding issue
Jul 31, 2020
4a874c7
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 31, 2020
d681bf8
simplify changes
Jul 31, 2020
10986bb
update user doc for pbi
Jul 31, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added sql-odbc/docs/user/img/pbi_disable_parallel_loading_tables.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added sql-odbc/docs/user/img/pbi_simple_graph.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 11 additions & 2 deletions sql-odbc/docs/user/power_bi_support.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Connecting Open Distro For ElasticSearch to Microsoft Power BI Desktop

**NOTE**: **The connector is under development. All connection options are not available yet. There could be issues while loading data**

## Prerequisites
* Microsoft Power BI Desktop
* [Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/docs/install/)
Expand All @@ -19,7 +17,14 @@

## Load Data

> **NOTE**: Currently only import mode is supported. Direct query support will be added soon.

* Open Power BI Desktop.

* Disable parallel loading of tables. Click on **Files** > **Options and settings** > **Options** > **CURRENT FILE** > **Data Load** > Deselect **Enable parallel loading of tables** and click **OK**.

<img src="img/pbi_disable_parallel_loading_tables.png" width="500">

* Click on **Home** > **Get Data** > **More** > **Other**. Select **Open Distro For Elasticsearch (Beta)**. Click on **Connect**.

<img src="img/pbi_select_connector.png" width="500">
Expand All @@ -42,6 +47,10 @@

* Click on **Load**.

* Select required columns for creating graph.

<img src="img/pbi_simple_graph.png">

## Troubleshooting

* If you get an following error, please install [Open Distro For Elasticsearch SQL ODBC Driver](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/odbc/).
Expand Down
21 changes: 20 additions & 1 deletion sql-odbc/src/PowerBIConnector/OdfeSqlOdbcPBIConnector.pq
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,20 @@ OdfeSqlOdbcPBIConnectorImpl = (Host as text, optional Port as number) as table =
Value.ReplaceType(toTable, Value.Type(source))
else
source,

SQLGetFunctions = Diagnostics.LogValue("SQLGetFunctions_Options", [
SQL_API_SQLBINDPARAMETER = false
]),

SqlCapabilities = Diagnostics.LogValue("SqlCapabilities_Options", [
SupportsTop = false,
LimitClauseKind = LimitClauseKind.LimitOffset,
SupportsNumericLiterals = true,
SupportsStringLiterals = true,
SupportsOdbcDateLiterals = true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you confirm these options for date/time/timestamp literals? Wondering if this refers to the {dt ...}-type syntax we've seen before, which ODFE SQL plugin doesn't support

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this refers to {d ..} syntax. But if we don't set this to true then date values will always be specified using Parameter Binding (https://docs.microsoft.com/en-us/power-query/odbc) which is also not supported.

I will check if it is possible to report the escape sequence syntax unsupported.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ODBC driver sets SQL_DATETIME_LITERALS to 0 in SQLGetInfo and a value of "0" means that SQL-92 datetime literals are not supported. (https://docs.microsoft.com/en-us/sql/odbc/reference/syntax/sqlgetinfo-function?view=sql-server-ver15)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified with @rupal-bq that timestamp values work with the current PBI connector; will look back at this when we include DirectQuery support

SupportsOdbcTimeLiterals = true,
SupportsOdbcTimestampLiterals = true
]),

OdbcDatasource = Odbc.DataSource(ConnectionString & Server & CredentialConnectionString & EncryptedConnectionString, [
// Do not view the tables grouped by their schema names.
Expand All @@ -123,6 +137,8 @@ OdfeSqlOdbcPBIConnectorImpl = (Host as text, optional Port as number) as table =
SQLColumns = SQLColumns,
SQLGetTypeInfo = SQLGetTypeInfo,
SQLGetInfo = SQLGetInfo,
SQLGetFunctions = SQLGetFunctions,
SqlCapabilities = SqlCapabilities,

OnError = OnOdbcError,

Expand Down Expand Up @@ -191,7 +207,10 @@ OdfeSqlOdbcPBIConnector.Publish = [
Category = "Other",
ButtonText = { Extension.LoadString("ButtonTitle"), Extension.LoadString("ButtonHelp") },
LearnMoreUrl = "https://opendistro.github.io/for-elasticsearch/",
SupportsDirectQuery = true,

// Disabling direct query due to limited SQL query support
SupportsDirectQuery = false,

SourceImage = OdfeSqlOdbcPBIConnector.Icons,
SourceTypeImage = OdfeSqlOdbcPBIConnector.Icons
];
Expand Down
Binary file modified sql-odbc/src/PowerBIConnector/bin/Release/OdfeSqlOdbcPBIConnector.mez
View file
Open in desktop
Binary file not shown.