[Feature/Reporting] Export Saved Search CSV as Dashboard Panel Action#34571
Merged
joelgriffith merged 83 commits intomasterfrom May 29, 2019
Merged
[Feature/Reporting] Export Saved Search CSV as Dashboard Panel Action#34571joelgriffith merged 83 commits intomasterfrom
joelgriffith merged 83 commits intomasterfrom
Conversation
tsullivan
changed the title
Contributor
💚 Build Succeeded |
tsullivan
force-pushed
the
feature/reporting/csv-export-panel-action
branch
from
edfb522 to
59034e0
Compare
Contributor
💚 Build Succeeded |
…/csv-export-panel-action
Contributor
💔 Build Failed |
Contributor
💚 Build Succeeded |
tsullivan
reviewed
Apr 9, 2019
x-pack/plugins/reporting/export_types/common/execute_job/get_custom_logo.ts
Show resolved
Hide resolved
tsullivan
reviewed
Apr 9, 2019
x-pack/plugins/reporting/export_types/csv/server/lib/flatten_hit.js
Outdated
Show resolved
Hide resolved
tsullivan
reviewed
Apr 9, 2019
x-pack/plugins/reporting/export_types/csv_from_savedobject/server/lib/generate_csv_search.ts
Show resolved
Hide resolved
tsullivan
reviewed
Apr 9, 2019
x-pack/plugins/reporting/export_types/csv_from_savedobject/server/lib/generate_csv_search.ts
Outdated
Show resolved
Hide resolved
tsullivan
reviewed
Apr 9, 2019
x-pack/plugins/reporting/export_types/csv_from_savedobject/server/lib/generate_csv_search.ts
Outdated
Show resolved
Hide resolved
tsullivan
reviewed
Apr 9, 2019
x-pack/plugins/reporting/export_types/csv_from_savedobject/server/lib/generate_csv_search.ts
Outdated
Show resolved
Hide resolved
tsullivan
reviewed
Apr 9, 2019
tsullivan
reviewed
Apr 9, 2019
x-pack/plugins/reporting/server/routes/generate_from_savedobject.ts
Outdated
Show resolved
Hide resolved
tsullivan
reviewed
Apr 9, 2019
x-pack/plugins/reporting/server/routes/generate_from_savedobject.ts
Outdated
Show resolved
Hide resolved
Contributor
💚 Build Succeeded |
tsullivan
reviewed
Apr 9, 2019
* Date field formatted via docvalue_fields; none else formatted * fix integration test fixtures * fix docvalue_fields search params for index patterns that have no time field * simplify * fix non-timebased index pattern test * self-review * fix lint
Contributor
💚 Build Succeeded |
* self review * rename types.d.ts to index * fix imports
tsullivan
reviewed
Apr 10, 2019
x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx
Outdated
Show resolved
Hide resolved
Contributor
💚 Build Succeeded |
…/csv-export-panel-action
This comment has been minimized.
This comment has been minimized.
added 2 commits
May 28, 2019 09:31
Contributor
Author
|
@kertal I think this is ready for another review pass (assuming we get a green CI) |
Contributor
💔 Build Failed |
Contributor
💚 Build Succeeded |
Member
|
@joelgriffith sorry for causing you timezone pain, maybe my inner clock wanted revenge for the Orlando - Vienna reverse Jetlag |
kertal
requested changes
May 29, 2019
Member
kertal
left a comment
kertal
left a comment
There was a problem hiding this comment.
just some notes I took before we talked, mainly it's not to show the download link when user's roles do not allow to generate the csv
| return searchEmbeddable.searchScope.searchSource.getSearchRequestBody(); | ||
| } | ||
|
|
||
| public isVisible = (panelActionAPI: PanelActionAPI): boolean => { |
Member
There was a problem hiding this comment.
this would mean checking for the reporting_user or superuser, right?
x-pack/plugins/reporting/server/routes/generate_from_savedobject.ts
Outdated
Show resolved
Hide resolved
Member
|
So I guess solution is to add an issue not to forget about it
Joel Griffith <notifications@github.com> schrieb am Mi. 29. Mai 2019 um
18:31:
… ***@***.**** commented on this pull request.
------------------------------
In x-pack/plugins/reporting/public/panel_actions/get_csv_panel_action.tsx
<#34571 (comment)>:
> + }
+
+ public async getSearchRequestBody({ searchEmbeddable }: { searchEmbeddable: any }) {
+ const adapters = searchEmbeddable.getInspectorAdapters();
+ if (!adapters) {
+ return {};
+ }
+
+ if (adapters.requests.requests.length === 0) {
+ return {};
+ }
+
+ return searchEmbeddable.searchScope.searchSource.getSearchRequestBody();
+ }
+
+ public isVisible = (panelActionAPI: PanelActionAPI): boolean => {
Unfortunately there doesn't appear to be a good way to do this as it
stands (from my chat with @kobelb <https://github.com/kobelb>). We have
an issue to migrate reporting over to the new privileges model which will
help address this in the future.
For the time being users will have to live with the Error notification if
they don't have proper access :(
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#34571?email_source=notifications&email_token=AADRH27PDCOZNPYF6RHYF5LPX2VXJA5CNFSM4HDVWIQKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOB2A2YMY#discussion_r288659503>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADRH2Z3NQA2NWVGZWB65VDPX2VXJANCNFSM4HDVWIQA>
.
|
Contributor
Author
|
Just to follow up for now:
|
kertal
approved these changes
May 29, 2019
Contributor
Author
|
Hello from the future! You're looking at history in the making! |
joelgriffith
pushed a commit
that referenced
this pull request
May 29, 2019
jkakavas
pushed a commit
to jkakavas/kibana
that referenced
this pull request
May 30, 2019
tsullivan
added a commit
that referenced
this pull request
Jan 11, 2023
## Summary This restores an endpoint that was added in 7.3 in [this PR](#34571), and was removed in 7.9 in [this PR](#71031). The changes are re-done on top of 7.17, but still has a mostly-compatible with the one that existed in 7.3-7.8. This serves 3rd parties that relied on the earlier experimental code. Supports: * Saved searches with filters * Saved searches with custom sorting * Saved searches with or without selected columns * Exports based on Index Patterns with or without a "time field" * Requests can have an [optional POST body](https://github.com/elastic/kibana/pull/148030/files#diff-0f565e26f3309c257fa919c5db227c3b7a78237015940c3d3677cbb1132a6701R27-R37) with extra time range filters and/or specify a custom time zone. LIMITATIONS: * This endpoint is currently not supported in 8.x at this time. * Saved Search objects created in older versions of Kibana may not work. * Searching across hundreds of shards in the query could cause Elasticsearch instability. * Some minor bugs in the output of the CSV may exist, such as fields not being formatted exactly as in the Discover table. * This code may be forward-ported to `main` in a way that uses a different API that is not compatible with this change. * Does not allow "raw state" to be merged with the Search object, as in the previous code. Otherwise, the API is compatible with the previous code. * This feature remains in "experimental" status, and is not ready to be documented at this time. ## Testing Since there is not a UI for this endpoint, there are a few options for testing: 1. Run the functional test: ```sh node scripts/functional_tests.js \ --config x-pack/test/reporting_api_integration/reporting_and_security.config.ts \ --grep 'CSV Generation from Saved Search ID' ``` 2. Create a saved search in Kibana, and use a script to send a request ```sh POST_URL="${HOST}/api/reporting/v1/generate/csv/saved-object/search:"$SAVED_SEARCH_ID ## Run transaction to generate a report, wait for execution completion, download the report, and send the # report as an email attachment # 1. Send a request to generate a report DOWNLOAD_PATH=$(curl --silent -XPOST "$POST_URL" -H "kbn-xsrf: kibana-reporting" -H "${AUTH_HEADER}" | jq -e -r ".payload.path | values") if [ -z "$DOWNLOAD_PATH" ]; then echo "Something went wrong! Could not send the request to generate a report!" 1>&2 # TEST curl --silent -XPOST "$POST_URL" -H "kbn-xsrf: kibana-reporting" -H "${AUTH_HEADER}" exit 1 fi # 2. Log the path used to download the report DOWNLOAD_PATH=${HOST}$DOWNLOAD_PATH echo Download path: $DOWNLOAD_PATH # 3. Wait for report execution to finish echo While the report is executing in the Kibana server, the reporting service will return a 503 status code response. STATUS='' while [[ -z $STATUS || $STATUS =~ .*503.* ]] do echo Waiting 5 seconds... sleep 5 STATUS=$(curl --silent --head "$DOWNLOAD_PATH" -H "${AUTH_HEADER}" | head -1) if [[ -z "$STATUS" || $STATUS =~ .*500.* ]]; then echo "Something went wrong! Could not request the report execution status!" 1>&2 curl "$DOWNLOAD_PATH" -H "${AUTH_HEADER}" 1>&2 exit 1 fi echo $STATUS done # 4. Download final report and show the contents in the console curl -v "$DOWNLOAD_PATH" -H "$AUTH_HEADER" ``` 3. Test that the above script from (2) works in 7.8, and continues to work after migrating to 7.17.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes: #18110
This changes introduces a new API (Thanks @tsullivan) to convert a saved-search object into an immediately-downloadable CSV! I'll be pushing some fixes and tweaks next week once I'm off of support ramp, with the goal of getting this merged soon-ish.
Testing
xpack.reporting.csv.enablePanelActionDownload: truein your config....panel action icon of the SavedSearch and clickDownload CSVChecklist
Use
strikethroughsto remove checklist items you don't feel are applicable to this PR.