feat: add event_logger to test_connection and create_database commands#13468
feat: add event_logger to test_connection and create_database commands#13468
Conversation
…into hugh/event-logger-refactor
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
…e/incubator-superset into hugh/event-logger-refactor
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
…into hugh/event-logger-refactor
…into hugh/event-logger-refactor
…into hugh/event-logger-refactor
…into hugh/use-event-db
betodealmeida
left a comment
There was a problem hiding this comment.
I left a few comments and mentioned a couple nits, but I have one concern with the rollback() that's being called inside the event_logger context manager. We should move it out and call it before logging, to ensure that it goes through even if logging fails.
| with event_logger( | ||
| action="db_connection_failed", | ||
| engine=database.db_engine_spec.__name__, | ||
| ): | ||
| db.session.rollback() | ||
| raise DatabaseConnectionFailedError() |
There was a problem hiding this comment.
For places where we do a rollback we should it outside the context manager, otherwise if the logging fails the transaction won't be rolled back:
| with event_logger( | |
| action="db_connection_failed", | |
| engine=database.db_engine_spec.__name__, | |
| ): | |
| db.session.rollback() | |
| raise DatabaseConnectionFailedError() | |
| db.session.rollback() | |
| event_logger.log_with_context( | |
| action="db_connection_failed", | |
| engine=database.db_engine_spec.__name__, | |
| ) | |
| raise DatabaseConnectionFailedError() |
Reversing the order ensures that even if the logging fails the rollback will still go through.
| with event_logger( | ||
| action=f"db_creation_failed.{ex.__class__.__name__}", | ||
| engine=database.db_engine_spec.__name__, | ||
| ): | ||
| raise DatabaseCreateFailedError() |
There was a problem hiding this comment.
Total nit, but in places where we just raise an exception inside the context manager like here and in lines 94-97 I think it would make more sense to use the log_with_context function instead:
| with event_logger( | |
| action=f"db_creation_failed.{ex.__class__.__name__}", | |
| engine=database.db_engine_spec.__name__, | |
| ): | |
| raise DatabaseCreateFailedError() | |
| event_logger.log_with_context( | |
| action=f"db_creation_failed.{ex.__class__.__name__}", | |
| engine=database.db_engine_spec.__name__, | |
| ) | |
| raise DatabaseCreateFailedError() |
| if database is None: | ||
| raise DBAPIError("Database is not found", None, None) |
There was a problem hiding this comment.
We can actually remove this and fix the type annotation in DatabaseDAO.build_db_for_connection_test, the class method never returns None.
tests/databases/commands_tests.py
Outdated
| def test_connection_db_exception( | ||
| self, mock_event_logger, mock_build_db_for_connection_test | ||
| ): | ||
| """Test that users can't export databases they don't have access to""" |
There was a problem hiding this comment.
Update docstring here and in the other 2 tests 😄
tests/databases/commands_tests.py
Outdated
| with self.assertRaises(DatabaseTestConnectionUnexpectedError): | ||
| command_without_db_name.run() |
There was a problem hiding this comment.
Now that we're using pytest, it's better to run tests using plain asserts instead, since it gives more context when the test fails. Also, we can check that the exception raised has the right error message:
| with self.assertRaises(DatabaseTestConnectionUnexpectedError): | |
| command_without_db_name.run() | |
| with pytest.raises(DatabaseTestConnectionUnexpectedError) as excinfo: | |
| command_without_db_name.run() | |
| assert str(excinfo.value) == ( | |
| "Unexpected error occurred, please check your logs for details" | |
| ) |
Same in the other 2 tests below.
| logger.exception(ex.exception) | ||
| event_logger.log_with_context( | ||
| action=f"db_creation_failed.{ex.__class__.__name__}", | ||
| engine=database.db_engine_spec.__name__, |
There was a problem hiding this comment.
@hughhhh what did you want to do about making the engine string consistent around each of these logs so that we can group by the same engine later when observing the logs?
There was a problem hiding this comment.
I ended up changing all the engine references in the event_logger.log_with_context to pull data from the database.db_engine_spec.__name__ I say we go with this for now and see the data then see if we need to create some mapping either in the service or downstream
apache#13468) Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

SUMMARY
Leverage refactored
event_loggerwith context managementDependent on #13441
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TEST PLAN
ADDITIONAL INFORMATION