Skip to content

Handling NULL values in Filters #825

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kkalyan wants to merge 12 commits into from
Closed

Handling NULL values in Filters #825

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
504e44e
null check
Jul 27, 2016
136139a
showing NULL
Jul 27, 2016
ab95f72
Merge branch 'master' into nullcheck
kkalyan Jul 28, 2016
a8a91ec
Merge branch 'master' into nullcheck
kkalyan Jul 29, 2016
72b619a
Merge branch 'master' into nullcheck
kkalyan Jul 31, 2016
0702dcd
Merge branch 'master' into nullcheck
kkalyan Aug 2, 2016
7462f65
Merge branch 'master' into nullcheck
kkalyan Aug 4, 2016
9deb84b
Merge branch 'master' into nullcheck
kkalyan Aug 8, 2016
4ce1517
NULL in filters
Aug 9, 2016
0ab4104
Merge branch 'master' into nullcheck
kkalyan Aug 9, 2016
1f7de43
Merge branch 'master' into nullcheck
kkalyan Aug 10, 2016
2650ec4
Merge branch 'master' into nullcheck
kkalyan Aug 15, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions caravel/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1410,9 +1410,14 @@ def get_filters(raw_filters):
if len(splitted) > 1:
for s in eq.split(','):
s = s.strip()
if s == 'NULL':
s = None
fields.append(Dimension(col) == s)
cond = Filter(type="or", fields=fields)
else:
#allows setting filter to query that value of dimension is NULL
if eq == 'NULL':
eq = None
cond = Dimension(col) == eq
if op == 'not in':
cond = ~cond
Expand Down
2 changes: 1 addition & 1 deletion caravel/viz.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ def get_df(self, query_obj=None):
if self.datasource.offset:
df.timestamp += timedelta(hours=self.datasource.offset)
df.replace([np.inf, -np.inf], np.nan)
df = df.fillna(0)
df = df.fillna('NULL')

@mistercrunch mistercrunch Jul 28, 2016

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this main create some problems where there's missing data expected to be numeric. Ultimately we want javascript's reserved null in the json. Would df.fillna(None) work?

@kkalyan kkalyan Jul 29, 2016 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

df.fillna(None) threw an error 'must specify a fill method or value'
But I get your point - if ('NULL') is treated as true in javascript. we may need some similar handling in javascript (like value=='NULL'?value:null).

Side note - should this fillna be done only for Dimensions and skip Metrics ?

return df

@property
Expand Down