Skip to content

Test for Importing saved objects from 6.x to 7.x #96102

Closed
bhavyarm wants to merge 2 commits intoelastic:7.xfrom
bhavyarm:savedObjects7.x
Closed

Test for Importing saved objects from 6.x to 7.x #96102
bhavyarm wants to merge 2 commits intoelastic:7.xfrom
bhavyarm:savedObjects7.x

Conversation

@bhavyarm
Copy link
Contributor

@bhavyarm bhavyarm commented Apr 1, 2021 •
edited
Loading

This is part 2 for making sure users can import saved objects between versions. Its importing saved objects from 6.8.x to 7.x.

This PR handled it for 7.x to 8.0.0 #91908

@bhavyarm
Copy link
Contributor Author

bhavyarm commented Apr 1, 2021 •
edited
Loading

@rudolf when I run this test in my local - its failing and crashing Kibana server and these are my logs. Can you please take a look? Thanks!

Server logs:

info [o.e.c.m.MetadataIndexStateService] [Bhavyas-MacBook-Pro.local] completed adding block write to indices [.kibana_pre6.5.0_001]
   │ proc [kibana] server    log   [17:36:15.591] [info][savedobjects-service] [.kibana] SET_SOURCE_WRITE_BLOCK -> CREATE_REINDEX_TEMP
   │ info [o.e.c.m.MetadataCreateIndexService] [Bhavyas-MacBook-Pro.local] [.kibana_7.13.0_reindex_temp] creating index, cause [api], templates [], shards [1]/[1]
   │ info [o.e.c.r.a.AllocationService] [Bhavyas-MacBook-Pro.local] updating number_of_replicas to [0] for indices [.kibana_7.13.0_reindex_temp]
   │ proc [kibana] Unhandled Promise rejection detected:
   │ proc [kibana] 
   │ proc [kibana] ResponseError: Saved object index alias [.kibana_7.13.0] not found: index_not_found_exception
   │ proc [kibana]     at onBody (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:337:23)
   │ proc [kibana]     at IncomingMessage.onEnd (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:264:11)
   │ proc [kibana]     at IncomingMessage.emit (events.js:327:22)
   │ proc [kibana]     at endReadableNT (internal/streams/readable.js:1327:12)
   │ proc [kibana]     at processTicksAndRejections (internal/process/task_queues.js:80:21) {
   │ proc [kibana]   meta: {
   │ proc [kibana]     body: { error: [Object], status: 404 },
   │ proc [kibana]     statusCode: 404,
   │ proc [kibana]     headers: {
   │ proc [kibana]       'content-type': 'application/json; charset=UTF-8',
   │ proc [kibana]       'content-length': '445'
   │ proc [kibana]     },
   │ proc [kibana]     meta: {
   │ proc [kibana]       context: null,
   │ proc [kibana]       request: [Object],
   │ proc [kibana]       name: 'elasticsearch-js',
   │ proc [kibana]       connection: [Object],
   │ proc [kibana]       attempts: 0,
   │ proc [kibana]       aborted: false
   │ proc [kibana]     }
   │ proc [kibana]   },
   │ proc [kibana]   isBoom: true,
   │ proc [kibana]   isServer: true,
   │ proc [kibana]   data: null,
   │ proc [kibana]   output: {
   │ proc [kibana]     statusCode: 500,
   │ proc [kibana]     payload: {
   │ proc [kibana]       message: 'An internal server error occurred',
   │ proc [kibana]       statusCode: 500,
   │ proc [kibana]       error: 'Internal Server Error'
   │ proc [kibana]     },
   │ proc [kibana]     headers: {}
   │ proc [kibana]   },
   │ proc [kibana]   [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/generalError'
   │ proc [kibana] }
   │ proc [kibana] 
   │ proc [kibana] Terminating process...
   │ proc [kibana]  server crashed  with status code 1

Test runner logs:

node scripts/functional_test_runner.js --grep 'should be able to import 6.8 saved objects into 7.x'
 debg Loading config file from "/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/x-pack/test/functional/config.js"
 debg Loading config file from "/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/test/common/config.js"
 debg Loading config file from "/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/test/functional/config.js"
 info Config loaded
 debg randomness seed: 1617312964895
 debg [webdriver] Creating session
 debg randomness seed: 1617312964923
 debg ===============creating roles and users===============
 debg creating role test_logstash_reader

 debg creating role global_canvas_all
 debg creating role global_discover_all
 debg creating role global_dashboard_read
 debg creating role global_discover_read
 debg creating role global_visualize_read
 debg creating role global_visualize_all
 debg creating role global_dashboard_all
 debg creating role global_maps_all
 debg creating role global_maps_read
 debg creating role geoshape_data_reader
 debg creating role antimeridian_points_reader
 debg creating role antimeridian_shapes_reader
 debg creating role meta_for_geoshape_data_reader
 info Remote initialized: chrome 89.0.4389.90
 info chromedriver version: 89.0.4389.23 (61b08ee2c50024bab004e48d2b1b083cdbdac579-refs/branch-heads/4389@{#294})
 debg creating role geoconnections_data_reader
 debg creating role geoall_data_writer
 debg creating role global_index_pattern_management_all
 debg creating role global_devtools_read
 debg creating role global_ccr_role
 debg creating role global_upgrade_assistant_role
 debg creating role manage_rollups_role
 debg creating role test_api_keys
 debg creating role manage_security
 debg creating role ccr_user
 debg creating role manage_ilm
 debg creating role index_management_user
 debg creating role ingest_pipelines_user
 debg creating role license_management_user
 debg creating role logstash_read_user
 debg creating role remote_clusters_user
 debg deleting user test_user
 debg no test user to delete
 debg default roles = superuser
 debg creating user test_user
 debg created user test_user
 debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC","visualization:visualize:legacyChartsLibrary":true}
 info Starting tests

 └-: Saved objects management
   └-> "before all" hook in "Saved objects management"
   └-: Export import saved objects between versions - 6.8.x -> 7.x
     └-> "before all" hook for "should be able to import 6.8 saved objects into 7.x"
     └-> should be able to import 6.8 saved objects into 7.x
       └-> "before each" hook: global before each for "should be able to import 6.8 saved objects into 7.x"
       └-> "before each" hook for "should be able to import 6.8 saved objects into 7.x"
         │ info [empty_kibana] Loading "mappings.json"
         │ info [empty_kibana] Deleted existing index ".kibana_task_manager_7.13.0_001"
         │ info [empty_kibana] Deleted existing index ".kibana_7.13.0_001"
         │ info [empty_kibana] Created index ".kibana"
         │ debg [empty_kibana] ".kibana" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
         │ debg Migrating saved objects
         │ERROR [migrate saved objects] request failed (attempt=1/5): socket hang up
         │ERROR [migrate saved objects] request failed (attempt=2/5): connect ECONNREFUSED 127.0.0.1:5620
         │ERROR [migrate saved objects] request failed (attempt=3/5): connect ECONNREFUSED 127.0.0.1:5620
         │ERROR [migrate saved objects] request failed (attempt=4/5): connect ECONNREFUSED 127.0.0.1:5620
         │ERROR [migrate saved objects] request failed (attempt=5/5): connect ECONNREFUSED 127.0.0.1:5620
         │ info Taking screenshot "/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/x-pack/test/functional/screenshots/failure/Saved objects management Export import saved objects between versions - 6_8_x -_ 7_x _before each_ hook for _should be able to import 6_8 saved objects into 7_x_.png"
         │ info Current URL is: data:/,
         │ info Saving page source to: /Users/bhavyarajumandya/Desktop/kibana_dev/kibana/x-pack/test/functional/failure_debug/html/Saved objects management Export import saved objects between versions - 6_8_x -_ 7_x _before each_ hook for _should be able to import 6_8 saved objects into 7_x_.html
         └- ✖ fail: Saved objects management Export import saved objects between versions - 6.8.x -> 7.x "before each" hook for "should be able to import 6.8 saved objects into 7.x"
         │      Error: [migrate saved objects] request failed (attempt=5/5): connect ECONNREFUSED 127.0.0.1:5620 -- and ran out of retries
         │       at KbnClientRequester.request (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_requester.ts:135:15)
         │       at processTicksAndRejections (internal/process/task_queues.js:93:5)
         │       at KbnClientSavedObjects.migrate (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_saved_objects.ts:100:22)
         │       at Object.migrateKibanaIndex (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/packages/kbn-es-archiver/src/lib/indices/kibana_index.ts:62:3)
         │       at Object.loadAction (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/packages/kbn-es-archiver/src/actions/load.ts:106:5)
         │       at EsArchiver.load (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/packages/kbn-es-archiver/src/es_archiver.ts:85:12)
         │       at EsArchiver.esArchiver.<computed> [as load] (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/test/common/services/kibana_server/extend_es_archiver.js:23:21)
         │       at Context.<anonymous> (test/functional/apps/saved_objects_management/import_saved_objects_between_versions_6.x_7.x.ts:25:7)
         │       at Object.apply (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)
         │ 
         │ 
         └-> "after all" hook for "should be able to import 6.8 saved objects into 7.x"
           │ERROR [GET http://elastic:changeme@localhost:5620/api/status] request failed (attempt=1/5): connect ECONNREFUSED 127.0.0.1:5620
           │ERROR [GET http://elastic:changeme@localhost:5620/api/status] request failed (attempt=2/5): connect ECONNREFUSED 127.0.0.1:5620
           │ERROR [GET http://elastic:changeme@localhost:5620/api/status] request failed (attempt=3/5): connect ECONNREFUSED 127.0.0.1:5620
           │ERROR [GET http://elastic:changeme@localhost:5620/api/status] request failed (attempt=4/5): connect ECONNREFUSED 127.0.0.1:5620
           │ERROR [GET http://elastic:changeme@localhost:5620/api/status] request failed (attempt=5/5): connect ECONNREFUSED 127.0.0.1:5620
           │ info Taking screenshot "/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/x-pack/test/functional/screenshots/failure/Saved objects management Export import saved objects between versions - 6_8_x -_ 7_x _after all_ hook for _should be able to import 6_8 saved objects into 7_x_.png"
           │ info Current URL is: data:/,
           │ info Saving page source to: /Users/bhavyarajumandya/Desktop/kibana_dev/kibana/x-pack/test/functional/failure_debug/html/Saved objects management Export import saved objects between versions - 6_8_x -_ 7_x _after all_ hook for _should be able to import 6_8 saved objects into 7_x_.html
           └- ✖ fail: Saved objects management Export import saved objects between versions - 6.8.x -> 7.x "after all" hook for "should be able to import 6.8 saved objects into 7.x"
           │      Error: [GET http://elastic:changeme@localhost:5620/api/status] request failed (attempt=5/5): connect ECONNREFUSED 127.0.0.1:5620 -- and ran out of retries
           │       at KbnClientRequester.request (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_requester.ts:135:15)
           │       at processTicksAndRejections (internal/process/task_queues.js:93:5)
           │       at KbnClientStatus.get (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_status.ts:44:22)
           │       at KbnClientPlugins.getEnabledIds (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_plugins.ts:20:21)
           │       at Object.unloadAction (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/packages/kbn-es-archiver/src/actions/unload.ts:42:27)
           │       at EsArchiver.unload (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/packages/kbn-es-archiver/src/es_archiver.ts:103:12)
           │       at EsArchiver.esArchiver.<computed> [as unload] (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/test/common/services/kibana_server/extend_es_archiver.js:23:21)
           │       at Context.<anonymous> (test/functional/apps/saved_objects_management/import_saved_objects_between_versions_6.x_7.x.ts:34:7)
           │       at Object.apply (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)
           │ 
           │ 
         └-> "after all" hook in "Saved objects management"
     │
     │0 passing (26.3s)
     │2 failing
     │
     │1)    Saved objects management
     │       Export import saved objects between versions - 6.8.x -> 7.x
     │         "before each" hook for "should be able to import 6.8 saved objects into 7.x":
     │
     │      Error: [migrate saved objects] request failed (attempt=5/5): connect ECONNREFUSED 127.0.0.1:5620 -- and ran out of retries
     │       at KbnClientRequester.request (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_requester.ts:135:15)
     │       at processTicksAndRejections (internal/process/task_queues.js:93:5)
     │       at KbnClientSavedObjects.migrate (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_saved_objects.ts:100:22)
     │       at Object.migrateKibanaIndex (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/packages/kbn-es-archiver/src/lib/indices/kibana_index.ts:62:3)
     │       at Object.loadAction (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/packages/kbn-es-archiver/src/actions/load.ts:106:5)
     │       at EsArchiver.load (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/packages/kbn-es-archiver/src/es_archiver.ts:85:12)
     │       at EsArchiver.esArchiver.<computed> [as load] (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/test/common/services/kibana_server/extend_es_archiver.js:23:21)
     │       at Context.<anonymous> (test/functional/apps/saved_objects_management/import_saved_objects_between_versions_6.x_7.x.ts:25:7)
     │       at Object.apply (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)
     │ 
     │ 
     │2)    Saved objects management
     │       Export import saved objects between versions - 6.8.x -> 7.x
     │         "after all" hook for "should be able to import 6.8 saved objects into 7.x":
     │
     │      Error: [GET http://elastic:changeme@localhost:5620/api/status] request failed (attempt=5/5): connect ECONNREFUSED 127.0.0.1:5620 -- and ran out of retries
     │       at KbnClientRequester.request (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_requester.ts:135:15)
     │       at processTicksAndRejections (internal/process/task_queues.js:93:5)
     │       at KbnClientStatus.get (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_status.ts:44:22)
     │       at KbnClientPlugins.getEnabledIds (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/kbn_client/kbn_client_plugins.ts:20:21)
     │       at Object.unloadAction (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/packages/kbn-es-archiver/src/actions/unload.ts:42:27)
     │       at EsArchiver.unload (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/packages/kbn-es-archiver/src/es_archiver.ts:103:12)
     │       at EsArchiver.esArchiver.<computed> [as unload] (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/test/common/services/kibana_server/extend_es_archiver.js:23:21)
     │       at Context.<anonymous> (test/functional/apps/saved_objects_management/import_saved_objects_between_versions_6.x_7.x.ts:34:7)
     │       at Object.apply (/Users/bhavyarajumandya/Desktop/kibana_dev/kibana/node_modules/@kbn/test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)
     │ 
     │

@TinaHeiligers TinaHeiligers added bug Fixes for quality problems that affect the customer experience Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// labels Apr 1, 2021
@bhavyarm
Copy link
Contributor Author

bhavyarm commented Apr 12, 2021

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

kibanamachine commented Apr 12, 2021 •
edited
Loading

💔 Build Failed

Failed CI Steps

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@bhavyarm
Copy link
Contributor Author

bhavyarm commented Apr 12, 2021

This one had too many commits and messed up main. closing and opened - #96869

@bhavyarm bhavyarm closed this Apr 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bug Fixes for quality problems that affect the customer experience Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t//

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@bhavyarm @kibanamachine @TinaHeiligers