-
Notifications
You must be signed in to change notification settings - Fork 13.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: improve performance on reports log queries #26416
fix: improve performance on reports log queries #26416
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #26416 +/- ##
==========================================
- Coverage 69.15% 67.04% -2.12%
==========================================
Files 1947 1947
Lines 76010 76032 +22
Branches 8481 8481
==========================================
- Hits 52565 50974 -1591
- Misses 21264 22877 +1613
Partials 2181 2181
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
superset/reports/models.py
Outdated
__table_args__ = ( | ||
Index("ix_report_execution_log_report_schedule_id", report_schedule_id), | ||
Index("ix_report_execution_log_start_dttm", start_dttm), | ||
Index("ix_report_execution_log_state", state), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This idx might have the opposite effect as the optimizer can potentially leverage it first when filtering on several fields, which will basically return all rows, since the status col has low cardinality
@dpgaspar do you have a sense how long this migration will take? Is there merit in adding a line item in |
Added details to the description and a line to UPDATING.md |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
(cherry picked from commit 2f3bb5b)
🏷️ preset:2024.8 |
SUMMARY
Add indexes to alerts and reports logs to improve REST API performance
Using test data with 100 reports with 10K report logs each spread out by 100 days. total 1M logs. Globally on 100 reports we have 10K logs per day.
Current PG explain for:
Other simpler example:
After creating the indexes
On the simpler example:
Observed improvement:
Locally upgrading took:
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TESTING INSTRUCTIONS
ADDITIONAL INFORMATION