Skip to content

Make ci compatible with os2.x #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 10, 2022
Merged

Make ci compatible with os2.x #153

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9c648bb
Fix the integration test to work with OpenSearch 2.x
deepdatta Jul 28, 2022
214f11e
Update the OpenSearch versions in .github/workflows/CI
deepdatta Aug 8, 2022
b04cad6
Install python in the logstash container for integration test to prev…
deepdatta Aug 9, 2022
f687832
Replace the python command to extract the version field with grep
deepdatta Aug 10, 2022
ba63570
Modify version check to account for OpenDistro
deepdatta Aug 10, 2022
99b9b54
Merge branch 'main' into make_ci_compatible_with_OS2.x
deepdatta Aug 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/CI.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
strategy:
matrix:
logstash: [ "7.16.3", "7.17.1" ]
opensearch: [ "1.2.1" ]
opensearch: [ "1.3.4", "2.1.0" ]
Copy link
Contributor

@sshivanii sshivanii Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we don't want the 1.2.1 version at all?

Copy link
Member

@VijayanB VijayanB Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other clients tests against all released version.

Copy link
Contributor Author

@deepdatta deepdatta Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we want to run the tests only on the latest released versions of 1.x and 2.x

secure: [ true, false ]

name: Integration Test logstash-output-opensearch against OpenSearch
Expand Down Expand Up @@ -78,4 +78,4 @@ jobs:

- name: Run Integration tests against OpenDistro-${{ matrix.opendistro }}-security-${{ matrix.secure }} for logstash ${{ matrix.logstash }}
run: |
./scripts/opendistro/docker-run.sh
./scripts/opendistro/docker-run.sh
4 changes: 3 additions & 1 deletion scripts/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
ARG LOGSTASH_VERSION
FROM docker.elastic.co/logstash/logstash-oss:${LOGSTASH_VERSION}
USER root
RUN apt-get update && apt-get install -y python
USER logstash
COPY --chown=logstash:logstash Gemfile /usr/share/plugins/plugin/Gemfile
COPY --chown=logstash:logstash *.gemspec VERSION* version* /usr/share/plugins/plugin/
Expand All @@ -10,4 +12,4 @@ RUN gem install bundler -v '< 2'
WORKDIR /usr/share/plugins/plugin
RUN bundle install --with test ci
COPY --chown=logstash:logstash . /usr/share/plugins/plugin
RUN bundle exec rake vendor
RUN bundle exec rake vendor
6 changes: 5 additions & 1 deletion spec/integration/outputs/compressed_indexing_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,11 @@
response = http_client.get("#{index_url}/_search?q=*&size=1000")
result = LogStash::Json.load(response.body)
result["hits"]["hits"].each do |doc|
expect(doc["_type"]).to eq(type)
if OpenSearchHelper.check_version?("< 2")
expect(doc["_type"]).to eq(type)
else
expect(doc).not_to include("_type")
end
expect(doc["_index"]).to eq(index)
end
end
Expand Down
6 changes: 5 additions & 1 deletion spec/integration/outputs/index_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,11 @@
response = http_client.get("#{index_url}/_search?q=*&size=1000")
result = LogStash::Json.load(response.body)
result["hits"]["hits"].each do |doc|
expect(doc["_type"]).to eq(type)
if OpenSearchHelper.check_version?("< 2")
expect(doc["_type"]).to eq(type)
else
expect(doc).not_to include("_type")
end
expect(doc["_index"]).to eq(index)
end
end
Expand Down
2 changes: 1 addition & 1 deletion spec/integration/outputs/parent_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
require_relative "../../../spec/opensearch_spec_helper"
require "logstash/outputs/opensearch"

context "join field tests", :integration => true do
context "join field tests", :integration => true && OpenSearchHelper.check_version?("<2") do

shared_examples "a join field based parent indexer" do
let(:index) { 10.times.collect { rand(10).to_s }.join("") }
Expand Down
10 changes: 10 additions & 0 deletions spec/opensearch_spec_helper.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,16 @@ def self.version
RSpec.configuration.filter[:version]
end

def self.check_version?(*requirement)
version = self.version
if version.nil? || version.empty?
puts "version tag isn't set. Returning false from 'check_version?`."
return false
end
release_version = Gem::Version.new(version).release
Gem::Requirement.new(requirement).satisfied_by?(release_version)
end

RSpec::Matchers.define :have_hits do |expected|
match do |actual|
expected == actual['hits']['total']['value']
Expand Down