Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix make_connection_configuration for MSSQL (adding a driver) #113

Merged
merged 6 commits into from
Aug 17, 2023

Conversation

TJaniF
Copy link
Contributor

@TJaniF TJaniF commented Jul 5, 2023

Hi :)

This came up in the GX Slack twice. Using the GXO with a MSSQL conn_id currently does not work because there is no driver being passed via the URI. If fails with:

sqlalchemy.exc.InterfaceError: (pyodbc.InterfaceError) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')

There was also an issue with the connection needing a database argument separate from the schema passed through the operator.

This PR should fix that. I added the possibility to pass a driver via the connection extras, if none is passed ODBC Driver 17 for SQL Server is passed, which is what worked for me when testing with the mcr.microsoft.com/mssql/server:2022-latest image.
I tested this change with Postgres and MSSQL. :)

@phanikumv I hope it is ok to tag you for review on this, Viraj told me you might be able to help :)

@TJaniF TJaniF requested a review from phanikumv July 5, 2023 16:07
@phanikumv phanikumv requested a review from Lee-W July 6, 2023 02:38
elif conn_type == "mysql":
odbc_connector = "mysql"
database_name = self.schema
else:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will conn_type = "mssql" or something similar be passed to this method? If so could we please try something like

elfi conn_type == "mssql":
    ...
else:
    raise ValueError(f"conn_type {conn_type} is not supported")

Copy link
Contributor Author

@TJaniF TJaniF Jul 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Lee-W, thank you for your review!
Yes, conn_type == "mssql" will be passed in. I adjusted the code with your suggestion. I might actually add something about odbc connection types in a second PR, but I will have to test that first :)

Copy link
Contributor

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@TJaniF
Copy link
Contributor Author

TJaniF commented Jul 10, 2023

@Lee-W apologies, I tested the operator again and found out that the connection only accepts the driver when a database is specified in the URI. Added the MSSQL default database master in case a user has none specified in their connection (which is possible because the connection works with the MsSqlOperator without naming the database). No other changes 😅

odbc_connector = "mssql+pyodbc"
uri_string = f"{odbc_connector}://{self.conn.login}:{self.conn.password}@{self.conn.host}:{self.conn.port}/{self.schema}" # noqa
ms_driver = self.conn.extra_dejson.get("driver") or "ODBC Driver 17 for SQL Server"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: not yet tested myself, but I think the following should work 🤔

Suggested change
ms_driver = self.conn.extra_dejson.get("driver") or "ODBC Driver 17 for SQL Server"
ms_driver = self.conn.extra_dejson.get("driver", "ODBC Driver 17 for SQL Server")

Copy link
Contributor

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good with this change. left a minor suggestion 🙂

@phanikumv phanikumv merged commit 09dbdac into astronomer:main Aug 17, 2023
@TJaniF
Copy link
Contributor Author

TJaniF commented Aug 18, 2023

@phanikumv thank you! 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants