Skip to content

Commit

Permalink
[Dev] enable opensearch snapshot for Darwin (opensearch-project#3537)
Browse files Browse the repository at this point in the history
Enable the downloading of Darwin for running the command
`yarn opensearch snapshot`.

Darwin is not officially supported but snapshots are being
built here:
https://build.ci.opensearch.org/job/distribution-build-opensearch/

Issue resolved:
opensearch-project#2944

Signed-off-by: Kawika Avilla <[email protected]>
Co-authored-by: Josh Romero <[email protected]>
  • Loading branch information
kavilla and joshuarrrr committed Mar 10, 2023
1 parent 3107769 commit e511bc4
Show file tree
Hide file tree
Showing 4 changed files with 46 additions and 9 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- [Windows] Add helper functions to work around the differences of platforms ([#2681](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2681))
- [Windows] Add `@osd/cross-platform` package to standardize path handling across platforms ([#2703](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2703))
- [Optimizer] Increase timeout waiting for the exiting of an optimizer worker ([#3193](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3193))

- [Darwin] Add support for Darwin for running OpenSearch snapshots with `yarn opensearch snapshot` ([#3537](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3537))

### 🐛 Bug Fixes

Expand Down
6 changes: 3 additions & 3 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ $ yarn osd clean

OpenSearch Dashboards requires a running version of OpenSearch to connect to. In a separate terminal you can run the latest snapshot built using:

_(Linux and Windows only - for MacOS, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
_(Linux, Windows, Darwin (MacOS) only - for others, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
```bash
$ yarn opensearch snapshot
```
Expand Down Expand Up @@ -184,7 +184,7 @@ By default, the snapshot command will run [a minimal distribution of OpenSearch]

If you would like to run OpenSearch with a particular plugin installed on the cluster snapshot, pass the `--P` flag after `yarn opensearch snapshot`. You can use the flag multiple times to install multiple plugins. The argument value can be a URL to the plugin's zip file, maven coordinates of the plugin, or a local zip file path (use `file://` followed by the absolute or relative path, in that case). For example:

_(Linux and Windows only - for MacOS, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
_(Linux, Windows, Darwin (MacOS) only - for others, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
```bash
$ yarn opensearch snapshot --P https://repo1.maven.org/maven2/org/opensearch/plugin/opensearch-test-plugin/2.4.0.0/opensearch-test-plugin-2.4.0.0.zip
```
Expand Down Expand Up @@ -214,7 +214,7 @@ $ yarn opensearch snapshot --version 2.2.0 -E cluster.name=test -E path.data=/tm

### Alternative - Run OpenSearch from tarball

OpenSearch does not yet create artifacts for MacOS, so you'll need to download, install, and run the tarball instead. (You'll also need Java installed and the `JAVA_HOME` environmental variable set - see [OpenSearch developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#install-prerequisites) for details).
If you would like to run OpenSearch from the tarball, you'll need to download the minimal distribution, install it, and then run the executable. (You'll also need Java installed and the `JAVA_HOME` environmental variable set - see [OpenSearch developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#install-prerequisites) for details).

1. Download the latest minimal distribution of OpenSearch from [the downloads page](https://opensearch.org/downloads.html#minimal). Note the version and replace in commands below.
2. Unzip the `tar.gz` file: `tar -xvf opensearch-<OpenSearch-version>-linux-x64.tar.gz`
Expand Down
5 changes: 3 additions & 2 deletions packages/osd-opensearch/src/artifact.js
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ const { createHash } = require('crypto');
const path = require('path');

const asyncPipeline = promisify(pipeline);
const SUPPORTED_PLATFORMS = ['linux', 'windows', 'darwin'];
const DAILY_SNAPSHOTS_BASE_URL = 'https://artifacts.opensearch.org/snapshots/core/opensearch';
// TODO: [RENAMEME] currently do not have an existing replacement
// issue: https://github.com/opensearch-project/OpenSearch-Dashboards/issues/475
Expand Down Expand Up @@ -184,8 +185,8 @@ async function getArtifactSpecForSnapshotFromUrl(urlVersion, log) {
const arch = process.arch === 'arm64' ? 'arm64' : 'x64';
const extension = process.platform === 'win32' ? 'zip' : 'tar.gz';

if (platform !== 'linux' && platform !== 'windows') {
throw createCliError(`Snapshots are only available for Linux and Windows`);
if (!SUPPORTED_PLATFORMS.includes(platform)) {
throw createCliError(`Snapshots are only available for Linux, Windows, and Darwin`);
}

const latestUrl = `${DAILY_SNAPSHOTS_BASE_URL}/${desiredVersion}-SNAPSHOT`;
Expand Down
42 changes: 39 additions & 3 deletions packages/osd-opensearch/src/artifact.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -164,17 +164,17 @@ describe('Artifact', () => {
});
});

it('should throw when on a non-Linux or non-Windows platform', async () => {
it('should throw when on a non-Linux, non-Windows, non-Darwin platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'darwin',
value: 'android',
},
arch: {
value: ORIGINAL_ARCHITECTURE,
},
});
await expect(Artifact.getSnapshot('default', 'INVALID_PLATFORM', log)).rejects.toThrow(
'Snapshots are only available for Linux'
'Snapshots are only available for Linux, Windows, and Darwin'
);
});

Expand All @@ -190,6 +190,42 @@ describe('Artifact', () => {
mockFetch(MOCKS.multipleArch[0]);
artifactTest();
});

it('should not throw when on a Linux platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'linux',
},
arch: {
value: 'x64',
},
});
artifactTest();
});

it('should not throw when on a Windows platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'win32',
},
arch: {
value: 'x64',
},
});
artifactTest();
});

it('should not throw when on a Darwin platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'darwin',
},
arch: {
value: 'x64',
},
});
artifactTest();
});
});

describe('with custom snapshot manifest URL', () => {
Expand Down

0 comments on commit e511bc4

Please sign in to comment.