Fix the segment interval for pulling metadata #3174
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Coverage remained the same at 69.175% when pulling a82d85f1834d2b5dc7a58257e53b0c7539fd4ee0 on Fokko:fd-fix-metadata into 40d9e15 on apache:master. |
3 similar comments
|
Coverage remained the same at 69.175% when pulling a82d85f1834d2b5dc7a58257e53b0c7539fd4ee0 on Fokko:fd-fix-metadata into 40d9e15 on apache:master. |
|
Coverage remained the same at 69.175% when pulling a82d85f1834d2b5dc7a58257e53b0c7539fd4ee0 on Fokko:fd-fix-metadata into 40d9e15 on apache:master. |
|
Coverage remained the same at 69.175% when pulling a82d85f1834d2b5dc7a58257e53b0c7539fd4ee0 on Fokko:fd-fix-metadata into 40d9e15 on apache:master. |
|
Coverage remained the same at 69.175% when pulling 3cbb114cbce4b582772f8098c2d1e204d7a7bc74 on Fokko:fd-fix-metadata into 40d9e15 on apache:master. |
2 similar comments
|
Coverage remained the same at 69.175% when pulling 3cbb114cbce4b582772f8098c2d1e204d7a7bc74 on Fokko:fd-fix-metadata into 40d9e15 on apache:master. |
|
Coverage remained the same at 69.175% when pulling 3cbb114cbce4b582772f8098c2d1e204d7a7bc74 on Fokko:fd-fix-metadata into 40d9e15 on apache:master. |
|
Hi guys and girls, I see the build has failed, but I'm pretty sure that the change I suggested does not affect the JavaScript build that is failing. Cheers, Fokko |
|
We've fixed the master branch, please rebase to get a green build. |
The end of the interval would be on the truncated today date, which means that you will exclude today. If your realtime ingestion job runs shorter than a day, the metadata cannot be pulled from the druid cluster.
|
@mistercrunch Thanks, I've just rebased against master. |
lxzhangchao
added a commit
to lxzhangchao/superset
that referenced
this pull request
Aug 2, 2017
Fixing the damn build (apache#3179) * Fixing the build * Going deeper [bugfix] only filterable columns should show up in FilterBox list (apache#3105) * [bugfix] only filterable columns should show up in FilterBox list * Touchups Datasource cannot be empty (apache#3035) add title description to model view (apache#3045) * add title description to model view * add missing import Add 'show/hide totals' option to pivot table vis (apache#3101) [bugfix] numeric value for date fields in table viz (apache#3036) Bug was present only when using the NOT GROUPED BY option fixes apache#3027 fix hive.fetch_logs (apache#2968) add Zalando to the list of organizations (apache#3171) docs: fixup installation examples code indentation (apache#3169) [bugfix] fix bar order (apache#3180) [bugfix] visualize flow error: 'Metric x is not valid' (apache#3181) The metric name in the frontend doesn't match the one generated on the backend. It turns out the explore view will default to the first metric so specifying one isn't needed. Fix the segment interval for pulling metadata (apache#3174) The end of the interval would be on the truncated today date, which means that you will exclude today. If your realtime ingestion job runs shorter than a day, the metadata cannot be pulled from the druid cluster. Bump cryptography to 1.9 (apache#3065) As 1.7.2 doesn't compile here with openssl 1.1.0f Escaping the user's SQL in the explore view (apache#3186) * Escaping the user's SQL in the explore view When executing SQL from SQL Lab, we use a lower level API to the database which doesn't require escaping the SQL. When going through the explore view, the stack chain leading to the same method may need escaping depending on how the DBAPI driver is written, and that is the case for Presto (and perhaps other drivers). * Using regex to avoid doubling doubles [sqllab] improve Hive support (apache#3187) * [sqllab] improve Hive support * Fix "Transport not open" bug * Getting progress bar to show * Bump pyhive to 0.4.0 * Getting [Track Job] button to show * Fix testzz Add BigQuery engine specifications (apache#3193) As contributed by @mxmzdlv on issue apache#945 [bugfix] fix merge conflict that broke Hive support (apache#3196) Adding 'apache' to docs (apache#3194) [druid] Allow custom druid postaggregators (apache#3146) * [druid] Allow custom druid postaggregators Also, fix the postaggregation for approxHistogram quantiles so it adds the dependent field and that can show up in the graphs/tables. In general, postAggregators add significant power, we should probably support including custom postAggregators. Plywood has standard postAggregators here, and a customAggregator escape hatch that allows you to define custom postAggregators. This commit adds a similar capability for Superset and a additional field/fields/fieldName breakdown of the typical naming for dependent aggregations, which should make it significantly easier to develop approxHistogram and custom postAggregation-required dashboards. * [druid] Minor style cleanup in tests file. * [druid] Apply code review suggestions * break out CustomPostAggregator into separate class. This just cleans up the creation of the postaggregator a little bit. * minor style issues. * move the function around so the git diff is more readable add combine config for metrics in pivot table (apache#3086) * add combine config for metrics in pivot table * change method to stack/unstack * update backendSync Autofocus search input in VizTypeControl modal onEnter (apache#2929) Speed up JS build time (apache#3203) Also bumping a few related libs JS Translation JS translations js translation fix issue 3204 (apache#3205) [bugfix] capture Hive job_id pre-url transformation (apache#3213) js translation fix issue 3204 (apache#3205) [bugfix] capture Hive job_id pre-url transformation (apache#3213) [docs] update url in CONTRIBUTING.md (apache#3212) [sqllab/cosmetics] add margin-top for labels in query history (apache#3222) [explore] nvd3 sort values in rich tooltip (apache#3197) [sqllab] fix UI shows 'The query returned no results' momentarily (apache#3214) this is visible when running async queries between the fetching and success state as the rows are getting cached in the component [explore] DatasourceControl to pick datasource in modal (apache#3210) * [explore] DatasourceControl to pick datasource in modal Makes it easier to change datasource, also makes it such that the list of all datasources doesn't need to be loaded upfront. * Adding more metadata
mistercrunch
pushed a commit
that referenced
this pull request
Sep 20, 2017
* Chinese page * Using react-intl-universal to improve multi language in react page * Using react-intl-universal to improve multi language in react page * react_intl_universal * change * change * change * change * change * change * change * merge * multiple page in js * merge * merge * merge * merge * Js Translations * JS Translation * JS Translations * Js translation * JS translations * JS translations * Js translaion * JS en Translation * JS Translation * upgrade document Fixing the damn build (#3179) * Fixing the build * Going deeper [bugfix] only filterable columns should show up in FilterBox list (#3105) * [bugfix] only filterable columns should show up in FilterBox list * Touchups Datasource cannot be empty (#3035) add title description to model view (#3045) * add title description to model view * add missing import Add 'show/hide totals' option to pivot table vis (#3101) [bugfix] numeric value for date fields in table viz (#3036) Bug was present only when using the NOT GROUPED BY option fixes #3027 fix hive.fetch_logs (#2968) add Zalando to the list of organizations (#3171) docs: fixup installation examples code indentation (#3169) [bugfix] fix bar order (#3180) [bugfix] visualize flow error: 'Metric x is not valid' (#3181) The metric name in the frontend doesn't match the one generated on the backend. It turns out the explore view will default to the first metric so specifying one isn't needed. Fix the segment interval for pulling metadata (#3174) The end of the interval would be on the truncated today date, which means that you will exclude today. If your realtime ingestion job runs shorter than a day, the metadata cannot be pulled from the druid cluster. Bump cryptography to 1.9 (#3065) As 1.7.2 doesn't compile here with openssl 1.1.0f Escaping the user's SQL in the explore view (#3186) * Escaping the user's SQL in the explore view When executing SQL from SQL Lab, we use a lower level API to the database which doesn't require escaping the SQL. When going through the explore view, the stack chain leading to the same method may need escaping depending on how the DBAPI driver is written, and that is the case for Presto (and perhaps other drivers). * Using regex to avoid doubling doubles [sqllab] improve Hive support (#3187) * [sqllab] improve Hive support * Fix "Transport not open" bug * Getting progress bar to show * Bump pyhive to 0.4.0 * Getting [Track Job] button to show * Fix testzz Add BigQuery engine specifications (#3193) As contributed by @mxmzdlv on issue #945 [bugfix] fix merge conflict that broke Hive support (#3196) Adding 'apache' to docs (#3194) [druid] Allow custom druid postaggregators (#3146) * [druid] Allow custom druid postaggregators Also, fix the postaggregation for approxHistogram quantiles so it adds the dependent field and that can show up in the graphs/tables. In general, postAggregators add significant power, we should probably support including custom postAggregators. Plywood has standard postAggregators here, and a customAggregator escape hatch that allows you to define custom postAggregators. This commit adds a similar capability for Superset and a additional field/fields/fieldName breakdown of the typical naming for dependent aggregations, which should make it significantly easier to develop approxHistogram and custom postAggregation-required dashboards. * [druid] Minor style cleanup in tests file. * [druid] Apply code review suggestions * break out CustomPostAggregator into separate class. This just cleans up the creation of the postaggregator a little bit. * minor style issues. * move the function around so the git diff is more readable add combine config for metrics in pivot table (#3086) * add combine config for metrics in pivot table * change method to stack/unstack * update backendSync Autofocus search input in VizTypeControl modal onEnter (#2929) Speed up JS build time (#3203) Also bumping a few related libs JS Translation JS translations js translation fix issue 3204 (#3205) [bugfix] capture Hive job_id pre-url transformation (#3213) js translation fix issue 3204 (#3205) [bugfix] capture Hive job_id pre-url transformation (#3213) [docs] update url in CONTRIBUTING.md (#3212) [sqllab/cosmetics] add margin-top for labels in query history (#3222) [explore] nvd3 sort values in rich tooltip (#3197) [sqllab] fix UI shows 'The query returned no results' momentarily (#3214) this is visible when running async queries between the fetching and success state as the rows are getting cached in the component [explore] DatasourceControl to pick datasource in modal (#3210) * [explore] DatasourceControl to pick datasource in modal Makes it easier to change datasource, also makes it such that the list of all datasources doesn't need to be loaded upfront. * Adding more metadata * Js translation * js tran * js trans * js trans * js tran * js trans * js trans * js tran * js translation * js trans * js translation * try load language pack async * Backend translations things * create language pack inside common data * performance improvement for js i18n. - js bundle should not contain localized content - we populate translation content from server-side, in boostrap.common.language_pack - in client-side, use promise to wrap around translation content. text will be translated after translation content arrived/parsed. - fix linting * fix Timer unit test * 1. add global hook for all tests, to make translation pack avaialble before each test starts. 2. fix unit test for Timer component 3. remove noused method get_locale, and modules 4. fix page reload after user change page language * parse and build i18n dictionary as a module * fix sync-backend task, which should run without DOM
timifasubaa
pushed a commit
to timifasubaa/incubator-superset
that referenced
this pull request
Oct 3, 2017
* Chinese page * Using react-intl-universal to improve multi language in react page * Using react-intl-universal to improve multi language in react page * react_intl_universal * change * change * change * change * change * change * change * merge * multiple page in js * merge * merge * merge * merge * Js Translations * JS Translation * JS Translations * Js translation * JS translations * JS translations * Js translaion * JS en Translation * JS Translation * upgrade document Fixing the damn build (apache#3179) * Fixing the build * Going deeper [bugfix] only filterable columns should show up in FilterBox list (apache#3105) * [bugfix] only filterable columns should show up in FilterBox list * Touchups Datasource cannot be empty (apache#3035) add title description to model view (apache#3045) * add title description to model view * add missing import Add 'show/hide totals' option to pivot table vis (apache#3101) [bugfix] numeric value for date fields in table viz (apache#3036) Bug was present only when using the NOT GROUPED BY option fixes apache#3027 fix hive.fetch_logs (apache#2968) add Zalando to the list of organizations (apache#3171) docs: fixup installation examples code indentation (apache#3169) [bugfix] fix bar order (apache#3180) [bugfix] visualize flow error: 'Metric x is not valid' (apache#3181) The metric name in the frontend doesn't match the one generated on the backend. It turns out the explore view will default to the first metric so specifying one isn't needed. Fix the segment interval for pulling metadata (apache#3174) The end of the interval would be on the truncated today date, which means that you will exclude today. If your realtime ingestion job runs shorter than a day, the metadata cannot be pulled from the druid cluster. Bump cryptography to 1.9 (apache#3065) As 1.7.2 doesn't compile here with openssl 1.1.0f Escaping the user's SQL in the explore view (apache#3186) * Escaping the user's SQL in the explore view When executing SQL from SQL Lab, we use a lower level API to the database which doesn't require escaping the SQL. When going through the explore view, the stack chain leading to the same method may need escaping depending on how the DBAPI driver is written, and that is the case for Presto (and perhaps other drivers). * Using regex to avoid doubling doubles [sqllab] improve Hive support (apache#3187) * [sqllab] improve Hive support * Fix "Transport not open" bug * Getting progress bar to show * Bump pyhive to 0.4.0 * Getting [Track Job] button to show * Fix testzz Add BigQuery engine specifications (apache#3193) As contributed by @mxmzdlv on issue apache#945 [bugfix] fix merge conflict that broke Hive support (apache#3196) Adding 'apache' to docs (apache#3194) [druid] Allow custom druid postaggregators (apache#3146) * [druid] Allow custom druid postaggregators Also, fix the postaggregation for approxHistogram quantiles so it adds the dependent field and that can show up in the graphs/tables. In general, postAggregators add significant power, we should probably support including custom postAggregators. Plywood has standard postAggregators here, and a customAggregator escape hatch that allows you to define custom postAggregators. This commit adds a similar capability for Superset and a additional field/fields/fieldName breakdown of the typical naming for dependent aggregations, which should make it significantly easier to develop approxHistogram and custom postAggregation-required dashboards. * [druid] Minor style cleanup in tests file. * [druid] Apply code review suggestions * break out CustomPostAggregator into separate class. This just cleans up the creation of the postaggregator a little bit. * minor style issues. * move the function around so the git diff is more readable add combine config for metrics in pivot table (apache#3086) * add combine config for metrics in pivot table * change method to stack/unstack * update backendSync Autofocus search input in VizTypeControl modal onEnter (apache#2929) Speed up JS build time (apache#3203) Also bumping a few related libs JS Translation JS translations js translation fix issue 3204 (apache#3205) [bugfix] capture Hive job_id pre-url transformation (apache#3213) js translation fix issue 3204 (apache#3205) [bugfix] capture Hive job_id pre-url transformation (apache#3213) [docs] update url in CONTRIBUTING.md (apache#3212) [sqllab/cosmetics] add margin-top for labels in query history (apache#3222) [explore] nvd3 sort values in rich tooltip (apache#3197) [sqllab] fix UI shows 'The query returned no results' momentarily (apache#3214) this is visible when running async queries between the fetching and success state as the rows are getting cached in the component [explore] DatasourceControl to pick datasource in modal (apache#3210) * [explore] DatasourceControl to pick datasource in modal Makes it easier to change datasource, also makes it such that the list of all datasources doesn't need to be loaded upfront. * Adding more metadata * Js translation * js tran * js trans * js trans * js tran * js trans * js trans * js tran * js translation * js trans * js translation * try load language pack async * Backend translations things * create language pack inside common data * performance improvement for js i18n. - js bundle should not contain localized content - we populate translation content from server-side, in boostrap.common.language_pack - in client-side, use promise to wrap around translation content. text will be translated after translation content arrived/parsed. - fix linting * fix Timer unit test * 1. add global hook for all tests, to make translation pack avaialble before each test starts. 2. fix unit test for Timer component 3. remove noused method get_locale, and modules 4. fix page reload after user change page language * parse and build i18n dictionary as a module * fix sync-backend task, which should run without DOM
michellethomas
pushed a commit
to michellethomas/panoramix
that referenced
this pull request
May 24, 2018
* Chinese page * Using react-intl-universal to improve multi language in react page * Using react-intl-universal to improve multi language in react page * react_intl_universal * change * change * change * change * change * change * change * merge * multiple page in js * merge * merge * merge * merge * Js Translations * JS Translation * JS Translations * Js translation * JS translations * JS translations * Js translaion * JS en Translation * JS Translation * upgrade document Fixing the damn build (apache#3179) * Fixing the build * Going deeper [bugfix] only filterable columns should show up in FilterBox list (apache#3105) * [bugfix] only filterable columns should show up in FilterBox list * Touchups Datasource cannot be empty (apache#3035) add title description to model view (apache#3045) * add title description to model view * add missing import Add 'show/hide totals' option to pivot table vis (apache#3101) [bugfix] numeric value for date fields in table viz (apache#3036) Bug was present only when using the NOT GROUPED BY option fixes apache#3027 fix hive.fetch_logs (apache#2968) add Zalando to the list of organizations (apache#3171) docs: fixup installation examples code indentation (apache#3169) [bugfix] fix bar order (apache#3180) [bugfix] visualize flow error: 'Metric x is not valid' (apache#3181) The metric name in the frontend doesn't match the one generated on the backend. It turns out the explore view will default to the first metric so specifying one isn't needed. Fix the segment interval for pulling metadata (apache#3174) The end of the interval would be on the truncated today date, which means that you will exclude today. If your realtime ingestion job runs shorter than a day, the metadata cannot be pulled from the druid cluster. Bump cryptography to 1.9 (apache#3065) As 1.7.2 doesn't compile here with openssl 1.1.0f Escaping the user's SQL in the explore view (apache#3186) * Escaping the user's SQL in the explore view When executing SQL from SQL Lab, we use a lower level API to the database which doesn't require escaping the SQL. When going through the explore view, the stack chain leading to the same method may need escaping depending on how the DBAPI driver is written, and that is the case for Presto (and perhaps other drivers). * Using regex to avoid doubling doubles [sqllab] improve Hive support (apache#3187) * [sqllab] improve Hive support * Fix "Transport not open" bug * Getting progress bar to show * Bump pyhive to 0.4.0 * Getting [Track Job] button to show * Fix testzz Add BigQuery engine specifications (apache#3193) As contributed by @mxmzdlv on issue apache#945 [bugfix] fix merge conflict that broke Hive support (apache#3196) Adding 'apache' to docs (apache#3194) [druid] Allow custom druid postaggregators (apache#3146) * [druid] Allow custom druid postaggregators Also, fix the postaggregation for approxHistogram quantiles so it adds the dependent field and that can show up in the graphs/tables. In general, postAggregators add significant power, we should probably support including custom postAggregators. Plywood has standard postAggregators here, and a customAggregator escape hatch that allows you to define custom postAggregators. This commit adds a similar capability for Superset and a additional field/fields/fieldName breakdown of the typical naming for dependent aggregations, which should make it significantly easier to develop approxHistogram and custom postAggregation-required dashboards. * [druid] Minor style cleanup in tests file. * [druid] Apply code review suggestions * break out CustomPostAggregator into separate class. This just cleans up the creation of the postaggregator a little bit. * minor style issues. * move the function around so the git diff is more readable add combine config for metrics in pivot table (apache#3086) * add combine config for metrics in pivot table * change method to stack/unstack * update backendSync Autofocus search input in VizTypeControl modal onEnter (apache#2929) Speed up JS build time (apache#3203) Also bumping a few related libs JS Translation JS translations js translation fix issue 3204 (apache#3205) [bugfix] capture Hive job_id pre-url transformation (apache#3213) js translation fix issue 3204 (apache#3205) [bugfix] capture Hive job_id pre-url transformation (apache#3213) [docs] update url in CONTRIBUTING.md (apache#3212) [sqllab/cosmetics] add margin-top for labels in query history (apache#3222) [explore] nvd3 sort values in rich tooltip (apache#3197) [sqllab] fix UI shows 'The query returned no results' momentarily (apache#3214) this is visible when running async queries between the fetching and success state as the rows are getting cached in the component [explore] DatasourceControl to pick datasource in modal (apache#3210) * [explore] DatasourceControl to pick datasource in modal Makes it easier to change datasource, also makes it such that the list of all datasources doesn't need to be loaded upfront. * Adding more metadata * Js translation * js tran * js trans * js trans * js tran * js trans * js trans * js tran * js translation * js trans * js translation * try load language pack async * Backend translations things * create language pack inside common data * performance improvement for js i18n. - js bundle should not contain localized content - we populate translation content from server-side, in boostrap.common.language_pack - in client-side, use promise to wrap around translation content. text will be translated after translation content arrived/parsed. - fix linting * fix Timer unit test * 1. add global hook for all tests, to make translation pack avaialble before each test starts. 2. fix unit test for Timer component 3. remove noused method get_locale, and modules 4. fix page reload after user change page language * parse and build i18n dictionary as a module * fix sync-backend task, which should run without DOM
mistercrunch
added
🏷️ bot
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi all,
When refreshing the Druid metadata. The end of the interval was truncated on today's date, e.g.
1901-01-01/2017-07-25, which means that you will exclude today according to ISO-8601. If, in our case, the realtime ingestion job runs shorter than a day, the metadata cannot be pulled from the druid cluster because today's segments will be excluded. This will change1901-01-01/2017-07-25into1901-01-01/2017-07-24T15:00:23.742353so we also include segments from today-midnight till now.This will fix #2350 and might fix #1816 and #2273, but the latter issues do not expose too much information about the actual underlying problem.
Cheers, Fokko