Broken link checker #938

zuochengding · 2021-11-15T21:00:15Z

Description

This PR is to add a new doc_link_checker automation workflow for preventing new broken/bad urls introducing to OSD. In addition, this PR will also fix some broken links existed in OSD today (caught by the tool) .

Note:

The PR depends on Add versioned doc support #905
The PR create a basic workflow and just fixes some of the reported broken links. Added a todo list in .lycheeexclude for the rest of unavailable urls/doc links.
lychee link checker doesn't have a good support for directory exclusion. So for any new dummy url introduced in the future, you should add it to .lycheeexclude specifically.

Issues Resolved

Verify broken links automatically #646

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

kavilla

AWESOME. Already catching bugs I see. So this relies on exclusive github thing right? I can't run this locally like before I create a PR?

Perhaps this is mimicking what the other repos did but what made us go with lychee?

.lycheeexclude

.github/workflows/doc_links_checker.yml

.lycheeexclude

.github/workflows/doc_links_checker.yml

zuochengding · 2021-11-15T23:38:30Z

AWESOME. Already catching bugs I see. So this relies on exclusive github thing right? I can't run this locally like before I create a PR?

Yes, It relies on Github Action workflow (listening to github action such as push / pull request on branch) and only serves as a last mile checker.

Perhaps this is mimicking what the other repos did but what made us go with lychee?

Yes, the same approach is used in opensearch as well. I just copied the similar script but changing the triggers and add more coverage. I don't know the original decision to go with lychee but also don't have any objection to use it with following reasons:

IMO, it is usually good to use the consistent tool across same project unless we know something much better.
It works good for purposes and with github action so that we don't need introduce it to our project dependencies.
The project is an actively maintained open source project. Here is comparison : https://github.com/lycheeverse/lychee#features

kavilla · 2021-11-16T00:31:35Z

Yes, It relies on Github Action workflow (listening to github action such as push / pull request on branch) and only serves as a last mile checker.

For sure. Is there any command I can run locally if I don't want to create a PR? More of a curiosity thing.

Yes, the same approach is used in opensearch as well. I just copied the similar script but changing the triggers and add more coverage. I don't know the original decision to go with lychee but also don't have any objection to use it with following reasons:

IMO, it is usually good to use the consistent tool across same project unless we know something much better.

It works good for purposes and with github action so that we don't need introduce it to our project dependencies.

The project is an actively maintained open source project. Here is comparison : https://github.com/lycheeverse/lychee#features

🤙🤙🤙 thanks!

zuochengding · 2021-11-16T07:05:55Z

For sure. Is there any command I can run locally if I don't want to create a PR? More of a curiosity thing.

Yes, we do, but you need to download the binary to your local environment.

They provide binaries for every release here: https://github.com/lycheeverse/lychee/releases
You can download the binary to your local, e.g.

wget -c https://github.com/lycheeverse/lychee/releases/download/v0.8.0/lychee-v0.8.0-x86_64-unknown-linux-gnu.tar.gz -O - | tar -xz

Then you can check any file locally with the binary, e.g.

lychee --no-progress --verbose ./apm.js

.lycheeexclude

src/plugins/console/server/lib/spec_definitions/js/ingest.ts

src/plugins/console/server/lib/spec_definitions/json/generated/cluster.pending_tasks.json

tmarkley · 2021-11-18T17:44:36Z

lychee link checker doesn't have a good support for directory exclusion. So for any new dummy url introduced in the future, you should add it to .lycheeexclude specifically.

Can we add this documentation to TESTING.md?

boktorbb

Looks good

.github/workflows/doc_links_checker.yml

.lycheeexclude

test/functional/apps/getting_started/_shakespeare.js

src/core/public/doc_links/doc_links_service.ts

src/plugins/console/server/lib/spec_definitions/json/generated/ping.json

src/plugins/console/server/lib/spec_definitions/json/generated/cluster.stats.json

src/plugins/home/server/tutorials/instructions/heartbeat_instructions.ts

TESTING.md

src/plugins/maps_legacy/public/map/service_settings.test.js

test/functional/apps/getting_started/_shakespeare.js

src/core/public/doc_links/doc_links_service.ts

src/core/server/logging/README.md

src/plugins/console/server/lib/spec_definitions/json/generated/cluster.pending_tasks.json

src/plugins/console/server/lib/spec_definitions/json/generated/cluster.stats.json

src/plugins/usage_collection/server/routes/stats/README.md

src/plugins/home/README.md

packages/osd-dev-utils/certs/README.md

tmarkley · 2021-11-30T23:01:14Z

@zuochengding after your next revision can you mark conversations as resolved if you've addressed them or this is no further action needed?

packages/osd-ace/src/ace/modes/lexer_rules/opensearch_sql_highlight_rules.ts

.github/workflows/doc_links_checker.yml

packages/osd-ace/src/ace/modes/lexer_rules/opensearch_sql_highlight_rules.ts

src/plugins/console/server/lib/spec_definitions/json/generated/indices.close.json

src/plugins/console/server/lib/spec_definitions/json/generated/nodes.info.json

src/plugins/console/server/lib/spec_definitions/json/generated/snapshot.create_repository.json

src/plugins/console/server/lib/spec_definitions/json/generated/snapshot.delete.json

src/plugins/console/server/lib/spec_definitions/json/generated/snapshot.delete_repository.json

src/plugins/console/server/lib/spec_definitions/json/generated/clear_scroll.json

src/plugins/home/server/services/sample_data/data_sets/ecommerce/saved_objects.ts

1. Fix broken links in OSD 2. Generate lycheeexcude list to filter out false negative warnings from test files or external links 3. Add TODO items for internal unavaiable links 4. Integrate with doc link service change. 5. Standardize all opensearch url with `https://opensearch.org/` and add unavilable urls into noDocument list Signed-off-by: Zuocheng Ding <[email protected]>

tmarkley

LGTM!

Old review.

1. Fix broken links in OSD 2. Generate lycheeexcude list to filter out false negative warnings from test files or external links 3. Add TODO items for internal unavaiable links 4. Integrate with doc link service change. 5. Standardize all opensearch url with `https://opensearch.org/` and add unavilable urls into noDocument list Signed-off-by: Zuocheng Ding <[email protected]>

* Fix Node selection when using built in node (#912) Fixed node selection to use the node executable instead of the user local node. Issue resolved: #908 Signed-off-by: Sven R <[email protected]> * [Release] add axios dependency to UI package This package was missing from release build so have to add it to this package. Brought in by custom branding. Signed-off-by: Kawika Avilla <[email protected]> * Add support for codeowners to repo (#935) Signed-off-by: Ryan Bogan <[email protected]> * [LIC] Allows the simplified header for new files (#936) * [LIC] Allows the simplified header for new files Signed-off-by: Ashwin Pc <[email protected]> * Updates licence variable names Signed-off-by: Ashwin Pc <[email protected]> Co-authored-by: Ashwin Pc <[email protected]> * [Git] update PR template (#937) Update template to remove javadoc check box since we do not have any javadocs and include how to run the tests we use to verify the build. Signed-off-by: Kawika Avilla <[email protected]> * Add release notes for Dashboards 1.2.0 (#944) This is the backport PR for #944 Signed-off-by: Neumann <[email protected]> * Add versioned document support in OSD This is PR is to add versioned document support in OSD. 1. Add logic to pick up doc version from package.json and convert it to `latest` if we are on default `main` branch. 2. Refactor doc_link_service to have 3 urls groups: opensearch, opensearchDashboards, and noDocumentation. 3. Update dynamic versioned doc links and clean up unused urls 4. Fix known url bug #769 5. Add unit tests for doclinks branch name conversion Signed-off-by: Zuocheng Ding <[email protected]> * [Branding] prevent logging when config not set (#941) Out of the box, the rendering service will check the config and see the default value and log an info message saying that the branding config is invalid or not set. Everytime you refresh the browser you will get those log messages. This sets it to only log error messages if the user sets the branding config and it is invalid. Include using default messages. Signed-off-by: Kawika Avilla <[email protected]> * [Version] Increment to 2.0 (#973) Version bump from 1.2 to 2.0 Signed-off-by: Kawika Avilla <[email protected]> * Add Lychee Link Checker into OSD (#938) 1. Fix broken links in OSD 2. Generate lycheeexcude list to filter out false negative warnings from test files or external links 3. Add TODO items for internal unavaiable links 4. Integrate with doc link service change. 5. Standardize all opensearch url with `https://opensearch.org/` and add unavilable urls into noDocument list Signed-off-by: Zuocheng Ding <[email protected]> * Fix Lychee Link Checker Error (#1011) Signed-off-by: Zuocheng Ding <[email protected]> * [CI] Add tests to github workflow Add unit tests to github workflow and also creating a "bad apples" environment variable. Some unit tests just fail on the CI for hardware issues. They should be improved but step one will be calling out the bad apples. Also due to the flakiness we can cache the previous run results and only run the tests that failed. It's too random to catch with the bad apples mechanism. But still added the continue on error for unit tests because it takes so long to re-run on the CI. So instead if it does fail we automatically echo there was a failure and ask them to re-run. However, if we can get permission for a github action that can add a comment to the PR then we could automatically add to PR. Next step will be improving. Also needed to limit the amount of workers because otherwise the hardware can't handle well so then it will accidentally create conflicts. This means we get an accurate test run but it is slower on the CI. Included integration tests which worked out of the box. Included e2e tests as well but it the chrome driver for the application was different from github's chrome so to run it I just upgraded it for the test run. Not ideal, ideally we should probably set up a docker env and install the specific versions since we are now depending on github's virtual env and the dependencies they installed there. But at least this is a first pace. Signed-off-by: Kawika Avilla <[email protected]> * Add bwc tests for osd with bundle (#871) tests include the following cases: verify default page work verify advanced savings work verify filter and query work Disable eslint check Add eslint-disable comment Revise license content in plugins and support Simplify filter and query test modify test name and fix PR comment update license header and remove env files fix timestamp issue update eslint and license Particailly Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * Add more bwc tests for osd without bundles (#900) This PR adds the following bwc tests: 1) verify sample data work properly for bwc 2) verify timeline visualization work properly for bwc This PR also simplifies check_filter_and_query bwc test. It first removes Unique Visitors check because even fix the time interval, the number of unique visitors number is random. Then it simplifies this bwc test. add more tests in check_timeline and modify test names change one query content to make bwc tests more robust update license header add missing test and solve timestamp issue fix eslint and comments Partially Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * Add bwc test data for osd without bundle (#927) This PR contains 13 zipped bwc test data for osd without bundle. The data has been tested by osd-1.1 and osd-1.2. To use, here are the steps: 1)unzip the data to opensearch, for example: tar -xvf odfe-1.13.2.tar.gz You need to remove data folder first if there is one in opensearch 2)run opensearch: ./bin/opensearch 3)run dashboards: ./bin/opensearch-dashboards 4)run any cypress test Partically Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * Add bwc test data for osd with bundle (#940) This PR contains 13 zipped bwc test data for osd with bundle. The data has been tested by osd-1.1 and osd-1.2. To use, here are the steps: 1)unzip the data to opensearch, for example: tar -xvf odfe-1.13.2.tar.gz You need to remove data folder first if there is one in opensearch 2)run opensearch: ./bin/opensearch 3)run dashboards: ./bin/opensearch-dashboards 4)run any cypress test Besides the above manual process, we now offer a script in this PR: #931 To run bwc test using osd bundle data, use this command: ./cypress/bwctest-osd.sh -o /path/to/opensearch.tar.gz -d /path/to/opensearch-dashboards.tar.gz -b true Pls see more details in the above PR. fix data issue for eCommerse data resubmit data to fix timestamp issue Partically Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * Add more bwc tests for osd with bundles (#901) This PR adds the following bwc tests: 1)verify sample data work properly for bwc 2)verify timeline visualization work properly for bwc add more commands check in check_timeline and rename sample data check minimize the login time and make the tests more robust change query content to make bwc test more robust update license header solve timestamp issue fix comments and eslint Partically Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * [Backwards Compatibility] restore URL forwarding from legacy app Forwarding legacy app to the current format of the application. This enables the usage of stored URLs and other links that referenced the format of the application URL that mentioned the application name. Since we changed the URL forwarding we changed this value and released. So incase forks were made and depended on this legacy formatted reference of the application. It will still work. There are also references of the application. Issue resolved: #1013 Signed-off-by: Kawika Avilla <[email protected]> * [draft] Node.js upgrade from 10 to 14 * precommit checks are failing with an error related to an lmdb-store dependency * unit tests are failing with an error related to lmdb-store * bumped react from 16.12 to 16.14 to resolve unmet peer dependencies, but we still need a resolution to remove the old version * need a replacement for @elastic/good Signed-off-by: Bishoy Boktor <[email protected]> * Adds transformIgnorePattern for lmdb-store and dependencies Signed-off-by: Bishoy Boktor <[email protected]> * Removes lmdb-store from transformIgnorePatterns and fixes unit test jest issues (#1012) Signed-off-by: Bishoy Boktor <[email protected]> * [CI] Add tests to github workflow Add unit tests to github workflow and also creating a "bad apples" environment variable. Some unit tests just fail on the CI for hardware issues. They should be improved but step one will be calling out the bad apples. Also due to the flakiness we can cache the previous run results and only run the tests that failed. It's too random to catch with the bad apples mechanism. But still added the continue on error for unit tests because it takes so long to re-run on the CI. So instead if it does fail we automatically echo there was a failure and ask them to re-run. However, if we can get permission for a github action that can add a comment to the PR then we could automatically add to PR. Next step will be improving. Also needed to limit the amount of workers because otherwise the hardware can't handle well so then it will accidentally create conflicts. This means we get an accurate test run but it is slower on the CI. Included integration tests which worked out of the box. Included e2e tests as well but it the chrome driver for the application was different from github's chrome so to run it I just upgraded it for the test run. Not ideal, ideally we should probably set up a docker env and install the specific versions since we are now depending on github's virtual env and the dependencies they installed there. But at least this is a first pace. Signed-off-by: Kawika Avilla <[email protected]> * Skips flaky server metrics collector tests * Adds `--no-deprecation` flag for integration tests caused by `shot` which is a downstream dependency of `hapi`. * The ServerMetricsCollector tests are flaky and rely on the existing v17 hapi library that Dashboards depends on. This will be upgraded for the 2.0 release along with the Node.js upgrade. Signed-off-by: Tommy Markley <[email protected]> * Update node version in github workflow to 14 Signed-off-by: Bishoy Boktor <[email protected]> * Bumps node to 14.18.2 and uses caret range for @types/node Signed-off-by: Bishoy Boktor <[email protected]> * Fix local merge issues Signed-off-by: Bishoy Boktor <[email protected]> Co-authored-by: Sven R <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Ryan Bogan <[email protected]> Co-authored-by: Ashwin P Chandran <[email protected]> Co-authored-by: Ashwin Pc <[email protected]> Co-authored-by: Sean Neumann <[email protected]> Co-authored-by: Zuocheng Ding <[email protected]> Co-authored-by: Anan <[email protected]> Co-authored-by: Tommy Markley <[email protected]>

1. Fix broken links in OSD 2. Generate lycheeexcude list to filter out false negative warnings from test files or external links 3. Add TODO items for internal unavaiable links 4. Integrate with doc link service change. 5. Standardize all opensearch url with `https://opensearch.org/` and add unavilable urls into noDocument list Signed-off-by: Zuocheng Ding <[email protected]>

* [Git] update PR template (#937) Update template to remove javadoc check box since we do not have any javadocs and include how to run the tests we use to verify the build. Signed-off-by: Kawika Avilla <[email protected]> * Add release notes for Dashboards 1.2.0 (#944) This is the backport PR for #944 Signed-off-by: Neumann <[email protected]> * Add versioned document support in OSD This is PR is to add versioned document support in OSD. 1. Add logic to pick up doc version from package.json and convert it to `latest` if we are on default `main` branch. 2. Refactor doc_link_service to have 3 urls groups: opensearch, opensearchDashboards, and noDocumentation. 3. Update dynamic versioned doc links and clean up unused urls 4. Fix known url bug #769 5. Add unit tests for doclinks branch name conversion Signed-off-by: Zuocheng Ding <[email protected]> * [Branding] prevent logging when config not set (#941) Out of the box, the rendering service will check the config and see the default value and log an info message saying that the branding config is invalid or not set. Everytime you refresh the browser you will get those log messages. This sets it to only log error messages if the user sets the branding config and it is invalid. Include using default messages. Signed-off-by: Kawika Avilla <[email protected]> * [Version] Increment to 2.0 (#973) Version bump from 1.2 to 2.0 Signed-off-by: Kawika Avilla <[email protected]> * Add Lychee Link Checker into OSD (#938) 1. Fix broken links in OSD 2. Generate lycheeexcude list to filter out false negative warnings from test files or external links 3. Add TODO items for internal unavaiable links 4. Integrate with doc link service change. 5. Standardize all opensearch url with `https://opensearch.org/` and add unavilable urls into noDocument list Signed-off-by: Zuocheng Ding <[email protected]> * Fix Lychee Link Checker Error (#1011) Signed-off-by: Zuocheng Ding <[email protected]> * [CI] Add tests to github workflow Add unit tests to github workflow and also creating a "bad apples" environment variable. Some unit tests just fail on the CI for hardware issues. They should be improved but step one will be calling out the bad apples. Also due to the flakiness we can cache the previous run results and only run the tests that failed. It's too random to catch with the bad apples mechanism. But still added the continue on error for unit tests because it takes so long to re-run on the CI. So instead if it does fail we automatically echo there was a failure and ask them to re-run. However, if we can get permission for a github action that can add a comment to the PR then we could automatically add to PR. Next step will be improving. Also needed to limit the amount of workers because otherwise the hardware can't handle well so then it will accidentally create conflicts. This means we get an accurate test run but it is slower on the CI. Included integration tests which worked out of the box. Included e2e tests as well but it the chrome driver for the application was different from github's chrome so to run it I just upgraded it for the test run. Not ideal, ideally we should probably set up a docker env and install the specific versions since we are now depending on github's virtual env and the dependencies they installed there. But at least this is a first pace. Signed-off-by: Kawika Avilla <[email protected]> * Add bwc tests for osd with bundle (#871) tests include the following cases: verify default page work verify advanced savings work verify filter and query work Disable eslint check Add eslint-disable comment Revise license content in plugins and support Simplify filter and query test modify test name and fix PR comment update license header and remove env files fix timestamp issue update eslint and license Particailly Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * Add more bwc tests for osd without bundles (#900) This PR adds the following bwc tests: 1) verify sample data work properly for bwc 2) verify timeline visualization work properly for bwc This PR also simplifies check_filter_and_query bwc test. It first removes Unique Visitors check because even fix the time interval, the number of unique visitors number is random. Then it simplifies this bwc test. add more tests in check_timeline and modify test names change one query content to make bwc tests more robust update license header add missing test and solve timestamp issue fix eslint and comments Partially Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * Add bwc test data for osd without bundle (#927) This PR contains 13 zipped bwc test data for osd without bundle. The data has been tested by osd-1.1 and osd-1.2. To use, here are the steps: 1)unzip the data to opensearch, for example: tar -xvf odfe-1.13.2.tar.gz You need to remove data folder first if there is one in opensearch 2)run opensearch: ./bin/opensearch 3)run dashboards: ./bin/opensearch-dashboards 4)run any cypress test Partically Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * Add bwc test data for osd with bundle (#940) This PR contains 13 zipped bwc test data for osd with bundle. The data has been tested by osd-1.1 and osd-1.2. To use, here are the steps: 1)unzip the data to opensearch, for example: tar -xvf odfe-1.13.2.tar.gz You need to remove data folder first if there is one in opensearch 2)run opensearch: ./bin/opensearch 3)run dashboards: ./bin/opensearch-dashboards 4)run any cypress test Besides the above manual process, we now offer a script in this PR: #931 To run bwc test using osd bundle data, use this command: ./cypress/bwctest-osd.sh -o /path/to/opensearch.tar.gz -d /path/to/opensearch-dashboards.tar.gz -b true Pls see more details in the above PR. fix data issue for eCommerse data resubmit data to fix timestamp issue Partically Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * Add more bwc tests for osd with bundles (#901) This PR adds the following bwc tests: 1)verify sample data work properly for bwc 2)verify timeline visualization work properly for bwc add more commands check in check_timeline and rename sample data check minimize the login time and make the tests more robust change query content to make bwc test more robust update license header solve timestamp issue fix comments and eslint Partically Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * [Backwards Compatibility] restore URL forwarding from legacy app Forwarding legacy app to the current format of the application. This enables the usage of stored URLs and other links that referenced the format of the application URL that mentioned the application name. Since we changed the URL forwarding we changed this value and released. So incase forks were made and depended on this legacy formatted reference of the application. It will still work. There are also references of the application. Issue resolved: #1013 Signed-off-by: Kawika Avilla <[email protected]> * [Link] Fix yarnpkg link error Issue: https://yarnpkg.com/latest.msi is unavailable now and will be rerouted to a 404 page. Add it to link checker allow list to unblock the PR process. Signed-off-by: Zuocheng Ding <[email protected]> * Use the OpenSearch Dashboards logo in the READMEs Signed-off-by: Tommy Markley <[email protected]> * Add .whitesource file to activate integration scan (#999) We already enable the access of WhiteSource integration with Github.com for this repo. However, the automatic PR of .whitesource is not created. We asked for the support from WhiteSource side and they suggested we could raise one by ourselves. This PR will also set the WhiteSource integration config mode to Local to be using the whitesource.config. Dashboards team can modify this configuration on their own to customize it. We are providing the one we had for all repos at this time. Issues Resolved opensearch-project/opensearch-build#721 * Add whitesource for to activate integration * Add links of documents for WhiteSource Signed-off-by: Zelin Hao <[email protected]> * [Build] remove legacy version check for plugin builds (#1029) Removes the SEMVAR check for external plugins. 7.9 is not relevant to the application. The semvar library was also preventing major.minor.patch.x which is the format from OpenSearch plugins. Related issue: #992 Signed-off-by: Kawika Avilla <[email protected]> * [Node 14] Upgrades Node version from 10.24.1 to 14.18.2 (#1028) * Addresses syntax changes between Node.js v10 and v14. * Bumps dependencies to address build/compatibility issues: * Bumps `@types/node` from v10.17.26 to v14.17.32 * Bumps `@elastic/good` from v8.1.1-kibana2 to v9.0.1-kibana3 * Bumps `react` from v16.12.0 to v16.14.0 * Bumps `@microsoft/api-documenter` from v7.7.2 to v7.13.65 * Bumps `@microsoft/api-extractor` from v7.7.0 to v7.18.17 * Bumps `@types/webpack` from v4.41.3 to v4.41.31 * Bumps `@types/webpack-env` from v1.15.2 to v1.16.3 * Bumps `sass-loader` from v8.0.2 to v10.2.0 * Bumps `lmdb-store` from v0.6.10 to v1.6.11 * Bumps `node-sass` from "sass/node-sass#v5" to v6.0.1 * Adds `--no-deprecation` flag for integration tests caused by `shot` which is a downstream dependency of `hapi`. * Skips flaky server metrics collector tests * The ServerMetricsCollector tests are flaky and rely on the existing v17 hapi library that Dashboards depends on. This will be upgraded for the 2.0 release along with the Node.js upgrade. (#1073) * Bumps react from 16.12 to 16.14 to resolve unmet peer dependencies, but we still need a resolution to remove the old version. * Adds transformIgnorePattern for weak-lru-cache and ordered-binary to fix unit test jest failures. * Refactors node cache to improve logging and separate databases Signed-off-by: Bishoy Boktor <[email protected]> Co-authored-by: Tommy Markley <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Ashwin Pc <[email protected]> * Add a script to run one command for all bwc tests (#931) Currently, even we have bwc tests and data, to run bwc, we need to copy and unzip data in opensearch, then run opensearch, dashboards and cypress. This script will add more automation to allow us use one command to run all the tests. Here is the cmd: ./scripts/bwctest-osd.sh -o /path/to/opensearch.tar.gz -d /path/to/opensearch-dashboards.tar.gz -v versions -b true/false -o is the path to the tested opensearch. Here we need to rename the folder to opensearch and zip it -d is the path to the tested opensearch-dashboards. Also need to rename the folder to opensearch-dashboards and zip it -v is the optional version para. You can specify one version or multiple versions like "odfe-1.1.0, osd-1.0.0". If no pass, it will run all the versions defined in the script. -b is the optional osd type para. If pass true, it will run osd bundle. If pass false, it will run osd vanilla. The default is false. update the usage section with new parameters add license header and move the script in scripts folder modify bwc test script: 1)use curl command to check the opensearch and dashboards status 2)create test groups to eliminate if clauses 3)modify var names 4)wrap each block into functions to make it more reusable 5)add more comments clean out usage on port 5601 add test command and modify checking logic fix license Partically Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * [Map] Remove hardcoded AWS paths Clean up temp aws paths in code base. Add a configurable flag `showRegionBlockedWarning` into map plugin level config file. Signed-off-by: Zuocheng Ding <[email protected]> * [Docs] remove invalid reference in CONVENTIONS.md (#1110) Removed missed reference in CONVENTIONS.md. Issue related: #1109 Signed-off-by: Kawika Avilla <[email protected]> * Upgrades babel, storybook, and postcss (#1104) * Upgrades dependencies to resolve react-dev-utils, browserslist, and postcss CVEs. * We have to stay on v6.3.x `@storybook` dependencies because of storybookjs/storybook#16837. 6.3.x still depends on older version of some of the `@babel` libraries. * The `autoprefixer` upgrade removes the browserslist warning during the build. * `css-loader`, `postcss-loader`, `postcss` upgrades were required to fix webpack errors. These upgrades contained a few breaking changes. * Minor version bumps to `react-router`, `react-router-dom`, and `styled-components` were done while troubleshooting bootstrap issues. Resolves #1055 Resolves #1094 Resolves #1095 Signed-off-by: Tommy Markley <[email protected]> * Bumps microsoft api-documenter and api-extractor (#1106) Resolves #1063 Signed-off-by: Tommy Markley <[email protected]> * Fixes incorrect license headers (#1131) Resolves #1130 Signed-off-by: Tommy Markley <[email protected]> * Fixes linting errors (#1115) Signed-off-by: Tommy Markley <[email protected]> * [Backwards Compatibility] update instructions in TESTING.md (#1030) fix PR comments Partially Resolved: opensearch-project/opensearch-build#705 Signed-off-by: Anan Zhuang <[email protected]> * [CI] upgrade to chromedriver 97 for github actions Github virtual env upgraded chrome: actions/runner-images#4861 Signed-off-by: Kawika Avilla <[email protected]> * Disable WhiteSource check fails on commits/PRs (#1149) * WhiteSource is not properly comparing scans against the latest changes in `main`. This prevents the need to override checks to merge PRs for those who don't have access (like the Dashboards Core members). * Cleans up the WhiteSource config file. We don't need gradle, maven, go, python, or ruby scans enabled. * Replaces the deprecated `ignoreSourceFiles` config with `fileSystemScan`. Resolves #1150 Signed-off-by: Tommy Markley <[email protected]> * Bump parse-link-header from 1.0.1 to 2.0.0 (#1108) Bumps [parse-link-header](https://github.com/thlorenz/parse-link-header) from 1.0.1 to 2.0.0. - [Release notes](https://github.com/thlorenz/parse-link-header/releases) - [Commits](thlorenz/parse-link-header@v1.0.1...v2.0.0) --- updated-dependencies: - dependency-name: parse-link-header dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Upgrades `hapi` from v17 to v20 (#1146) * Bumping 3 major versions ahead introduces many breaking changes. Hapi provides a detailed changelog: https://hapi.dev/resources/changelog * v18 notes: hapijs/hapi#3871 * v19 notes: hapijs/hapi#4017 * Bumps `raw-loader` from v3.1.0 to v4.0.2 to address a bootstrap warning. No breaking changes other than bumping Node.js to v10. * Removes the `--no-deprecation` flag for the integration tests since the newest version of hapi doesn't use the deprecated library. Resolves #1070 Resolves #1073 Resolves #1076 Resolves #1088 Resolves #1090 Signed-off-by: Tommy Markley <[email protected]> * [BUG] fix disableWelcomeScreen config (#1143) disableWelcomeScreen was erroneously removed from being exposed to browser (for testing purposes) and was not able to pass the config to disable the welcome screen showing. Issue: #1138 Signed-off-by: Kawika Avilla <[email protected]> * [Tests] configurable skip checksum verification (#1207) This enables configuring the skipping of checksum verification for integration and functional tests. The out-of-box experience enables tests to pull down an artifact of OpenSearch to run frontend tests against. However, if there was an issue with the publishing of the checksum, for example: opensearch-project/opensearch-build#1497 Then any CI for OpenSearch Dashboards is severely blocked. This lets the out-of-box experience get around this. This shouldn't be used permenantly and should be toggled off when no longer blocked. Issue resolved: #1205 Signed-off-by: Kawika Avilla <[email protected]> * Bump nanoid from 3.1.30 to 3.2.0 (#1173) Bumps [nanoid](https://github.com/ai/nanoid) from 3.1.30 to 3.2.0. - [Release notes](https://github.com/ai/nanoid/releases) - [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md) - [Commits](ai/nanoid@3.1.30...3.2.0) --- updated-dependencies: - dependency-name: nanoid dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Don't terminate the server on NodeDeprecationWarning (#1185) The last AWS SDK for Javascript that supports Node 10 (v3.45.0) emits a NodeDeprecationWarning to indicate that Node 10 is no longer supported. Without this workaround, this crashes the OSD server, so it becomes impossible to interact with other AWS services from within OSD (e.g., in a custom plugin) until the Node 14 upgrade is done. Signed-off-by: Thilo-Alexander Ginkel <[email protected]> * Removes KUI Generator and related dependencies (#1105) * KUI is deprecated and we will not be adding new components. * Removes all dependencies that are no longer used in the package. * Updates the README to reflect the deprecation path. * Removes the create and document component scripts as well as the remaining references to generator-kui. Resolves #1059 Resolves #1061 Signed-off-by: Tommy Markley <[email protected]> * Bump markdown-it from 10.0.0 to 12.3.2 (#1140) Bumps [markdown-it](https://github.com/markdown-it/markdown-it) from 10.0.0 to 12.3.2. - [Release notes](https://github.com/markdown-it/markdown-it/releases) - [Changelog](https://github.com/markdown-it/markdown-it/blob/master/CHANGELOG.md) - [Commits](markdown-it/markdown-it@10.0.0...12.3.2) --- updated-dependencies: - dependency-name: markdown-it dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix createStateContainerReactHelpers documentation (#1213) Issues Resolved: #1197 Signed-off-by: Thilo-Alexander Ginkel <[email protected]> * Bumps `node-fetch` from v2.6.1 to v2.6.7 (#1169) Resolves #1162 Signed-off-by: Tommy Markley <[email protected]> * Removes deprecated `request` and `@percy/agent` (#1113) * Addresses first set of dependencies that are upstream from `[email protected]`. There is more work to do but a webpack upgrade is required first. * Replaces usage of `request` in integration tests with `tough-cookie`. * `@percy/agent` is deprecated and is replaced by `@percy/cli`. * Bumps `follow-redirects` to address CVE. * Also removes unnecessary user management logic from functional tests. Resolves #1133 Incremental change towards addressing #1066 Signed-off-by: Tommy Markley <[email protected]> * [Bug] fix incorrect import for opensearch aggs (#1192) Incorrect import statement that was introduced here: #688 Verified other imports and the rest look fine. Issue: n/a Signed-off-by: Kawika Avilla <[email protected]> * Re-enable WhiteSource check fails on commits/PRs (#1226) * WhiteSource seems to have resolved the previous bug, and we are still able to merge even if the check fails. Signed-off-by: Tommy Markley <[email protected]> * Removes storybook package and related code (#1172) In order to address potential licensing issues as well as resolve related CVEs, all storybook code is removed. The storybook features have been broken since the fork and the work to fix everything was greater than removing it. Alternatives will be considered in the future. Resolves #1130 Resolves #1171 Signed-off-by: Tommy Markley <[email protected]> * Run build and test workflow on all branches (#1222) * Skips feature branches * Use the `.nvmrc` file for the `node` version instead of a hard-coded version. Resolves #1023 Signed-off-by: Tommy Markley <[email protected]> * Initial Drag and Drop plugin code (#946) * Initial Drag and Drop plugin code Signed-off-by: Ashwin Pc <[email protected]> * Adds state management to Drag and Drop Signed-off-by: Ashwin Pc <[email protected]> * Moves Drag and Drop to create visualization menu Signed-off-by: Ashwin Pc <[email protected]> * Field Search in Data panel (#995) Add ability to search on index fields Signed-off-by: Abbas Hussain <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Sean Neumann <[email protected]> Co-authored-by: Zuocheng Ding <[email protected]> Co-authored-by: Anan <[email protected]> Co-authored-by: Tommy Markley <[email protected]> Co-authored-by: Zelin Hao <[email protected]> Co-authored-by: Bishoy Boktor <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Thilo-Alexander Ginkel <[email protected]> Co-authored-by: Abbas Hussain <[email protected]>

zuochengding added docs Improvements or additions to documentation enhancement New feature or request rename-docs Document rename tracking labels Nov 15, 2021

zuochengding requested review from tmarkley, kavilla, boktorbb and ananzh November 15, 2021 21:00

zuochengding self-assigned this Nov 15, 2021

kavilla reviewed Nov 15, 2021

View reviewed changes

kavilla linked an issue Nov 15, 2021 that may be closed by this pull request

Verify broken links automatically #646

Closed

tmarkley reviewed Nov 18, 2021

View reviewed changes

.lycheeexclude Outdated Show resolved Hide resolved

src/plugins/console/server/lib/spec_definitions/js/ingest.ts Show resolved Hide resolved

src/plugins/console/server/lib/spec_definitions/json/generated/cluster.pending_tasks.json Outdated Show resolved Hide resolved

boktorbb previously approved these changes Nov 18, 2021

View reviewed changes

ananzh previously requested changes Nov 18, 2021

View reviewed changes

zuochengding dismissed boktorbb’s stale review via 75398f0 November 19, 2021 20:44

zuochengding force-pushed the broken-link-checker branch from e8f44bb to 75398f0 Compare November 19, 2021 20:44

zuochengding requested a review from a team as a code owner November 19, 2021 20:44

zuochengding requested review from tmarkley, boktorbb, ananzh and kavilla November 19, 2021 21:01

kavilla reviewed Nov 22, 2021

View reviewed changes

src/plugins/console/server/lib/spec_definitions/json/generated/cluster.stats.json Outdated Show resolved Hide resolved

kavilla reviewed Nov 22, 2021

View reviewed changes

src/plugins/home/server/tutorials/instructions/heartbeat_instructions.ts Outdated Show resolved Hide resolved

TESTING.md Outdated Show resolved Hide resolved

tmarkley reviewed Nov 23, 2021

View reviewed changes

src/plugins/maps_legacy/public/map/service_settings.test.js Outdated Show resolved Hide resolved

tmarkley reviewed Nov 23, 2021

View reviewed changes

test/functional/apps/getting_started/_shakespeare.js Outdated Show resolved Hide resolved

tmarkley reviewed Nov 23, 2021

View reviewed changes

packages/osd-dev-utils/certs/README.md Outdated Show resolved Hide resolved

zuochengding force-pushed the broken-link-checker branch from 75398f0 to 55eb0d4 Compare December 1, 2021 01:12

kavilla previously approved these changes Dec 1, 2021

View reviewed changes

tmarkley reviewed Dec 2, 2021

View reviewed changes

tmarkley added the v1.3.0 label Dec 3, 2021

zuochengding dismissed kavilla’s stale review via 2cc1dd9 December 3, 2021 22:07

zuochengding force-pushed the broken-link-checker branch from 55eb0d4 to 2cc1dd9 Compare December 3, 2021 22:07

kavilla approved these changes Dec 7, 2021

View reviewed changes

tmarkley approved these changes Dec 8, 2021

View reviewed changes

zuochengding merged commit 8c11eff into opensearch-project:main Dec 8, 2021

kavilla linked an issue Dec 22, 2021 that may be closed by this pull request

[BUG] Dev Tools 'Open Documentation' link leads to non-existent page. #1041

Closed

kavilla mentioned this pull request Dec 22, 2021

[BUG] Dev Tools 'Open Documentation' link leads to non-existent page. #1041

Closed

zuochengding mentioned this pull request Dec 22, 2021

[1.x] [Doc] Add versioned doc support and lychee link checker #1042

Merged

7 tasks

tmarkley mentioned this pull request Jan 4, 2022

[BUG] Replace documents with prod urls #335

Closed

kavilla mentioned this pull request Jan 13, 2022

[BUG] broken link behind wrench in Dev-tools/console #1137

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Broken link checker #938

Broken link checker #938

zuochengding commented Nov 15, 2021 •

edited

Loading

kavilla left a comment •

edited

Loading

zuochengding commented Nov 15, 2021 •

edited

Loading

kavilla commented Nov 16, 2021 •

edited

Loading

zuochengding commented Nov 16, 2021

tmarkley commented Nov 18, 2021

boktorbb left a comment

tmarkley commented Nov 30, 2021

tmarkley left a comment

Broken link checker #938

Broken link checker #938

Conversation

zuochengding commented Nov 15, 2021 • edited Loading

Description

Issues Resolved

Check List

kavilla left a comment • edited Loading

Choose a reason for hiding this comment

zuochengding commented Nov 15, 2021 • edited Loading

kavilla commented Nov 16, 2021 • edited Loading

zuochengding commented Nov 16, 2021

tmarkley commented Nov 18, 2021

boktorbb left a comment

Choose a reason for hiding this comment

tmarkley commented Nov 30, 2021

tmarkley left a comment

Choose a reason for hiding this comment

zuochengding commented Nov 15, 2021 •

edited

Loading

kavilla left a comment •

edited

Loading

zuochengding commented Nov 15, 2021 •

edited

Loading

kavilla commented Nov 16, 2021 •

edited

Loading