Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

changes required for using to Power BI Service with Open Distro For Elasticsearch #669

Merged
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
427bd79
pbi connector
Jul 19, 2020
401a053
update names
Jul 19, 2020
11856cd
- add mez file
Jul 20, 2020
e693269
nit
Jul 20, 2020
4158974
nit
Jul 20, 2020
586793f
image resize
Jul 20, 2020
9cb48a4
resize image
Jul 20, 2020
c234b88
remove extra lines
Jul 20, 2020
0d0c050
nit
Jul 20, 2020
50d4ccf
update mez file
Jul 20, 2020
023fb67
rename connector
Jul 21, 2020
0b857c0
rename mez file
Jul 21, 2020
06ae486
- add basic & aws auth
Jul 23, 2020
b2ff222
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 23, 2020
3f6ff5d
add connector
Jul 23, 2020
ad73d91
update doc
Jul 23, 2020
91d5cbe
add ssl for auth
Jul 23, 2020
ddc725b
update docs for errors
Jul 23, 2020
b946351
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 23, 2020
2bd944c
- remove sqlgetinfo & sqlcapabilities values from connector
Jul 24, 2020
ed0afb9
adding EncryptedConnectionString
Jul 24, 2020
22dc963
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 28, 2020
82e9a72
add data type for columns
Jul 28, 2020
5022f1f
add alternate column size
Jul 28, 2020
f4c7068
reverting changes in catalog test
Jul 28, 2020
a76c382
nit
Jul 28, 2020
debe0c5
revert changes for column size validation
Jul 28, 2020
6add9da
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 28, 2020
a80eee8
address PR comments
Jul 29, 2020
e6490e4
Update SQL_GROUP_BY value for fixing query folding issue
Jul 29, 2020
e1032c3
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 29, 2020
9a36d47
removing AstVisitor for resolving query folding issue with data type …
Jul 30, 2020
8fa089b
fix parameter binding issue
Jul 31, 2020
4a874c7
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Jul 31, 2020
d681bf8
simplify changes
Jul 31, 2020
10986bb
update user doc for pbi
Jul 31, 2020
bb4913d
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Aug 4, 2020
37584d8
fix query folding issue while applying filter
Aug 5, 2020
446f986
change literal prefix & suffix value
Aug 5, 2020
c4f54e9
add user doc for using Power BI Service
Aug 6, 2020
96e47c0
resize image
Aug 6, 2020
73f265a
- add server instead of host & port for Power BI Service
Aug 7, 2020
4594d99
resize image
Aug 7, 2020
39bb952
resize image
Aug 7, 2020
c4f6fde
resize image
Aug 7, 2020
93aca03
Merge branch 'develop' of https://github.com/rupal-bq/sql into featur…
Aug 7, 2020
4eccc8e
update connector file
Aug 7, 2020
a492ce3
update docs for aws auth credentials
Aug 10, 2020
cc70ce4
fix typo
Aug 10, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified sql-odbc/docs/user/img/pbi_connection_string_options.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified sql-odbc/docs/user/img/pbi_error_conn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added sql-odbc/docs/user/img/pbi_gateway_status.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added sql-odbc/docs/user/img/pbi_publish_report.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added sql-odbc/docs/user/img/pbi_publish_status.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added sql-odbc/docs/user/img/pbi_select_workspace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added sql-odbc/docs/user/img/pbi_service_setting.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
80 changes: 80 additions & 0 deletions sql-odbc/docs/user/power_bi_service_support.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Connecting Open Distro For ElasticSearch to Microsoft Power BI Service

## Setup
* Download and Install [On-premises data gateway](https://docs.microsoft.com/en-us/data-integration/gateway/service-gateway-install)
* Change the path for custom data connector folder in On-premises data gateway so that the gateway can find the custom connector.
> NOTE: Ensure the gateway service account has permissions to access the custom connector folder.

<img src="img/pbi_gateway_connector_path.png" width=500>

* Verify the status of data gateway is ready to be used.

<img src="img/pbi_gateway_status.png" width=500>

* Login to Power BI Service.
* Click on **Setting** > **Manage Gateway**.

<img src="img/pbi_service_setting.png" width=400>

* Select **Allow user's custom data connectors to refresh through this gateway cluster(preview)**. Click on **Apply**.

<img src="img/pbi_service_cluster_setting.png">

* Click on **Add data sources to use the gateway**.
* Select Data Source Type as **Open Distro For Elasticsearch**.
* Enter Data Source Name and Server values.
* Select required **Authentication Method**. **Anonymous** refers to auth **NONE** and **Key** refers to auth **AWS_SIGV4**.
* Select Encryption mode for connection.
* Click on **Add**.

<img src="img/pbi_service_data_source.png" width=650>

* You will get a **Connection Successful** message.

<img src="img/pbi_service_data_source_success.png" width=650>


## Publish Report

* Follow [instructions](./power_bi_support.md) to create graph using Open Disto For Elasticsearch Data connector.
* Click on **Publish** to publish the report on Power BI service.

<img src="img/pbi_publish_report.png">

* Select destination and click on **Select**.

<img src="img/pbi_select_workspace.png" width=400>

* You will get a success message when report is published.

<img src="img/pbi_publish_status.png" width=400>

* Click on **Open '%report name%' in Power BI** to open published report in Power BI service.

## Modify report using Power BI Service

* Click on **Edit report** to modfify report.

<img src="img/pbi_service_edit_reoprt.png">

* Use **Filters**,**Visualizations** and **Fields** to modify report.

<img src="img/pbi_service_modified_report.png">

## Dataset Scheduled Refresh

* Click on **Settings** > **Datasets**.
* Select required Gateway and click on **Apply**.

<img src="img/pbi_service_dataset_gateway.png" width=800>

* Turn on Keep your data up to date option.
* Select refresh frequency and timezone.
* Add email for failure notifications if required.
* Click on **Apply**.

<img src="img/pbi_service_dataset_refresh.png" width=550>

* You can also check history by clicking on **Refresh history**.

<img src="img/pbi_service_dataset_refresh_history.png" width=550>
2 changes: 1 addition & 1 deletion sql-odbc/docs/user/power_bi_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@

<img src="img/pbi_third_party_warning.png" width="500">

* Enter host and port values. Click on **OK**.
* Enter server value. Click on **OK**.

<img src="img/pbi_connection_string_options.png" width="500">

Expand Down
40 changes: 11 additions & 29 deletions sql-odbc/src/PowerBIConnector/OdfeSqlOdbcPBIConnector.pq
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,17 @@ shared OdfeSqlOdbcPBIConnector.Contents = Value.ReplaceType(OdfeSqlOdbcPBIConnec

// Wrapper function to provide additional UI customization.
OdfeSqlOdbcPBIConnectorType = type function (
Host as (type text meta [
Documentation.FieldCaption = "Host",
Server as (type text meta [
Documentation.FieldCaption = "Server",
Documentation.FieldDescription = "The hostname of the Open Distro For Elasticsearch server.",
Documentation.SampleValues = { "localhost" }
]),
optional Port as (type number meta [
Documentation.FieldCaption = "Port",
Documentation.FieldDescription = "The port of the Open Distro For Elasticsearch server is running on.",
Documentation.SampleValues = { 9200 }
Documentation.SampleValues = { "localhost:9200" }
])
)
as table meta [
Documentation.Name = "Open Distro For Elasticsearch"
];

OdfeSqlOdbcPBIConnectorImpl = (Host as text, optional Port as number) as table =>
OdfeSqlOdbcPBIConnectorImpl = (Server as text) as table =>
let
Credential = Extension.CurrentCredential(),
AuthenticationMode = Credential[AuthenticationKind],
Expand Down Expand Up @@ -61,21 +56,9 @@ OdfeSqlOdbcPBIConnectorImpl = (Host as text, optional Port as number) as table =
UseSSL = 0
],

// Set host & port in connection string.
// Do not include port in connection string for aws server connection.
Server =
if Port <> null then
[
Host = Host,
Port = Port
]
else
[
Host = Host
],

ConnectionString = [
Driver = "ODFE SQL ODBC Driver"
Driver = "ODFE SQL ODBC Driver",
Host = Server
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to do anything on the driver side to handle this change?
(ie. will the driver understand a Host value of localhost:9200?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The driver doesn't need any change. I tested with PBI and ODBC administrator.

],

SQLGetInfo = Diagnostics.LogValue("SQLGetInfo_Options", [
Expand Down Expand Up @@ -123,7 +106,7 @@ OdfeSqlOdbcPBIConnectorImpl = (Host as text, optional Port as number) as table =
SupportsOdbcTimestampLiterals = true
]),

OdbcDatasource = Odbc.DataSource(ConnectionString & Server & CredentialConnectionString & EncryptedConnectionString, [
OdbcDatasource = Odbc.DataSource(ConnectionString & CredentialConnectionString & EncryptedConnectionString, [
// Do not view the tables grouped by their schema names.
HierarchicalNavigation = false,
// Prevents execution of native SQL statements. Extensions should set this to true.
Expand Down Expand Up @@ -154,7 +137,7 @@ OdfeSqlOdbcPBIConnectorImpl = (Host as text, optional Port as number) as table =
OnOdbcError = (errorRecord as record) =>
let
ErrorMessage = errorRecord[Message],
ConnectionHostPort = errorRecord[Detail][DataSourcePath],
ConnectionServer = errorRecord[Detail][DataSourcePath],

IsDriverNotInstalled = Text.Contains(ErrorMessage, "doesn't correspond to an installed ODBC driver"),

Expand All @@ -168,7 +151,7 @@ OnOdbcError = (errorRecord as record) =>
if IsDriverNotInstalled then
error Error.Record("", "The Open Distro For Elasticsearch SQL ODBC driver is not installed. Please install the driver")
else if IsHostUnreachable then
error Error.Record("", "Couldn't reach server. Please double-check the host, port and auth.")
error Error.Record("", "Couldn't reach server. Please double-check the server and auth. " & ConnectionServer)
else
error errorRecord;

Expand All @@ -178,10 +161,9 @@ OdfeSqlOdbcPBIConnector = [
TestConnection = (dataSourcePath) =>
let
json = Json.Document(dataSourcePath),
Host = json[Host],
Port = json[Port]
Server = json[Server]
in
{ "OdfeSqlOdbcPBIConnector.Contents", Host, Port },
{ "OdfeSqlOdbcPBIConnector.Contents", Server },

// Authentication modes
Authentication = [
Expand Down
Binary file not shown.