Skip to content

Order by clause with Hive causes performance degradation. #454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
grafke wants to merge 4 commits into from
Closed

Order by clause with Hive causes performance degradation. #454

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5491da7
main_metric_expr may not always be present in the subselect in which …
May 10, 2016
11ab62e
main_metric_expr in some cases can be eg. SUM(column_name) in which c…
May 10, 2016
e791bff
dont really need the main_metric_expr
May 10, 2016
713d895
optimized unused import
May 10, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 1 addition & 11 deletions caravel/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
from six import string_types
from sqlalchemy import (
Column, Integer, String, ForeignKey, Text, Boolean, DateTime, Date,
Table, create_engine, MetaData, desc, select, and_, func)
Table, create_engine, MetaData, select, and_, func)
from sqlalchemy.engine import reflection
from sqlalchemy.ext.declarative import declared_attr
from sqlalchemy.orm import relationship
Expand Down Expand Up @@ -583,13 +583,6 @@ def query( # sqla
m.sqla_col
for m in self.metrics if m.metric_name in metrics]

if metrics:
main_metric_expr = [
m.sqla_col for m in self.metrics
if m.metric_name == metrics[0]][0]
else:
main_metric_expr = literal_column("COUNT(*)").label("ccount")

select_exprs = []
groupby_exprs = []

Expand Down Expand Up @@ -670,16 +663,13 @@ def query( # sqla
else:
qry = qry.where(and_(*where_clause_and))
qry = qry.having(and_(*having_clause_and))
if groupby:
qry = qry.order_by(desc(main_metric_expr))
qry = qry.limit(row_limit)

if timeseries_limit and groupby:
subq = select(inner_select_exprs)
subq = subq.select_from(tbl)
subq = subq.where(and_(*(where_clause_and + inner_time_filter)))
subq = subq.group_by(*inner_groupby_exprs)
subq = subq.order_by(desc(main_metric_expr))
subq = subq.limit(timeseries_limit)
on_clause = []
for i, gb in enumerate(groupby):
Expand Down