Enable data client with sample data server side

[kristenTian]

Description

Backend support for adding Sample data when multiple data source enabled.

What to expect:

• Can install sample data to arbitrary data source, which includes:
  • Create index in cluster
  • Create saved object in dashboard
• Create saved objects with datasource_id pre-appended
• Created Dashboard, visualization and search with datasource title appended

Issues Resolved

The backend section for: #4240

Screenshot

The demo shows that sample data is installed to a data source. Proved by that the local cluster does not have the index.

Note: There are a few vis not showing due to the types are not supported by multiple data source feature yet.

Screen.Recording.2023-06-12.at.1.34.16.PM.mov

Related

#4278
#1990

Testing the changes

Check List

• All tests pass
  • yarn test:jest
  • yarn test:jest_integration
  • yarn test:ftr
• New functionality includes testing.
• New functionality has been documented.
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[codecov]

Codecov Report

Merging #4268 (9b0c644) into main (55b293a) will decrease coverage by 0.03%.
The diff coverage is 76.59%.

❗ Current head 9b0c644 differs from pull request most recent head 7406970. Consider uploading reports for the commit 7406970 to get more accurate results

@@            Coverage Diff             @@
##             main    #4268      +/-   ##
==========================================
- Coverage   66.36%   66.33%   -0.03%     
==========================================
  Files        3229     3245      +16     
  Lines       62163    62423     +260     
  Branches     9629     9665      +36     
==========================================
+ Hits        41252    41407     +155     
- Misses      18585    18669      +84     
- Partials     2326     2347      +21     

Flag	Coverage Δ
Linux	?
Windows	66.33% <76.59%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ns/home/server/services/sample_data/routes/list.ts	52.63% <57.14%> (ø)
.../services/sample_data/data_sets/ecommerce/index.ts	75.00% <66.66%> (ø)
...erver/services/sample_data/data_sets/logs/index.ts	75.00% <66.66%> (ø)
...home/server/services/sample_data/data_sets/util.ts	79.41% <79.41%> (ø)
...home/server/services/sample_data/routes/install.ts	67.16% <93.33%> (ø)
...gins/data_source/server/client/configure_client.ts	71.42% <100.00%> (ø)
...ta_source/server/legacy/configure_legacy_client.ts	71.15% <100.00%> (ø)
...er/services/sample_data/data_sets/flights/index.ts	100.00% <100.00%> (ø)
...me/server/services/sample_data/routes/uninstall.ts	76.19% <100.00%> (ø)

... and 16 files with indirect coverage changes

[kristenTian]

cypress, 2, 6 ,9-12

Unrelated failure tests.

[zengyan-amazon]

shall we add some tests?

[zengyan-amazon]

we can still have auto expand replica to 0-1

[kristenTian]

During local test, I notice this number varies base on how many data nodes my data source have. - e.g. If single node, the number needs to be 0-3. Otherwise, it will error out with: "expected max cap on auto expand to be a multiple of total awareness attributes [3]" from: https://github.com/opensearch-project/OpenSearch/blob/743eed4c8ecb11c4898d340c0127c84347df5b40/server/src/main/java/org/opensearch/cluster/routing/allocation/AwarenessReplicaBalance.java#L109

Seems 2 options available, 1)set it to 0-all or 2) ignore the setting making it default to disabled.

[zengyan-amazon]

I am ok to set to no replica for sample data then.

[zengyan-amazon]

let's add a try catch here in case the get data source operation fails.

shall we do this before inserting data into remote data source? and fail this API call if the get data source fails

[seraphjiang]

@kristenTian
could not find how to choose from Video, are we using default data source concept? maybe i missed?

[kristenTian]

@kristenTian could not find how to choose from Video, are we using default data source concept? maybe i missed?

The UI part is not included in this PR. Only backend implementation is included.

[kristenTian]

shall we add some tests?

Originally there were no UT related to these files. Added them.