Skip to content

added cloudwatch style contains operator #5219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Swiddis merged 5 commits into from
Mar 23, 2026
Merged

added cloudwatch style contains operator #5219

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ad44495
added cloudwatch style contains operator
ThyTran1402 Mar 9, 2026
852e5fb
added escape \ before wrapping with % and udpated condition.md, index.md
ThyTran1402 Mar 10, 2026
f5b3bbf
cleanup
ThyTran1402 Mar 16, 2026
ac8c74f
cleanup and fixed doctests and integ tests
ThyTran1402 Mar 20, 2026
0dc8e95
added and registered ilike()
ThyTran1402 Mar 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions integ-test/src/test/java/org/opensearch/sql/ppl/WhereCommandIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,34 @@ public void testLikeOperatorCaseInsensitive() throws IOException {
verifyDataRows(result3, rows("Amber"));
}

@Test
public void testContainsOperator() throws IOException {
JSONObject result =
executeQuery(
String.format(
"source=%s | where firstname contains 'mbe' | fields firstname",

@LantaoJin LantaoJin Mar 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the contains case-sensitive or insensitive? Need more tests for this.
Can you link the cloudwatch doc link in description?
And this PR should include user document updates. maybe in condition.md

@ThyTran1402 ThyTran1402 Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think contains is case-insensitive since it's implemented with ILIKE with %value% wrapping.
If a user writes firstname contains '%', the wrapped pattern becomes %%% which would match everything, which is likely unintended. I'll fix this and add more test cases.
Here is the doc provided: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax-Filter.html

TEST_INDEX_ACCOUNT));
verifyDataRows(result, rows("Amber"));

result =
executeQuery(
String.format(
"source=%s | where firstname contains 'zzz' | fields firstname",
TEST_INDEX_ACCOUNT));
assertEquals(0, result.getInt("total"));
}

@Test
public void testContainsOperatorCaseInsensitive() throws IOException {
// contains uses ilike semantics - case insensitive
JSONObject result =
executeQuery(
String.format(
"source=%s | where firstname contains 'MBE' | fields firstname",
TEST_INDEX_ACCOUNT));
verifyDataRows(result, rows("Amber"));
}

@Test
public void testIsNullFunction() throws IOException {
JSONObject result =
Expand Down
1 change: 1 addition & 0 deletions language-grammar/src/main/antlr4/OpenSearchPPLLexer.g4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,7 @@ REDUCE: 'REDUCE';

// BOOL FUNCTIONS
LIKE: 'LIKE';
CONTAINS: 'CONTAINS';
ISNULL: 'ISNULL';
ISNOTNULL: 'ISNOTNULL';
BETWEEN: 'BETWEEN';
Expand Down
1 change: 1 addition & 0 deletions language-grammar/src/main/antlr4/OpenSearchPPLParser.g4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -945,6 +945,7 @@ geoIpProperty
| GREATER
| NOT_GREATER
| REGEXP
| CONTAINS
;

singleFieldRelevanceFunctionName
Expand Down
1 change: 1 addition & 0 deletions ppl/src/main/antlr/OpenSearchPPLLexer.g4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -459,6 +459,7 @@ CAST: 'CAST';
// BOOL FUNCTIONS
LIKE: 'LIKE';
ILIKE: 'ILIKE';
CONTAINS: 'CONTAINS';
ISNULL: 'ISNULL';
ISNOTNULL: 'ISNOTNULL';
CIDRMATCH: 'CIDRMATCH';
Expand Down
2 changes: 2 additions & 0 deletions ppl/src/main/antlr/OpenSearchPPLParser.g4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1444,6 +1444,7 @@ positionFunctionName
| REGEXP
| LIKE
| ILIKE
| CONTAINS
;

singleFieldRelevanceFunctionName
Expand Down Expand Up @@ -1609,6 +1610,7 @@ searchableKeyWord
| ELSE
| ARROW
| BETWEEN
| CONTAINS
| EXISTS
| SOURCE
| INDEX
Expand Down
9 changes: 9 additions & 0 deletions ppl/src/main/java/org/opensearch/sql/ppl/parser/AstExpressionBuilder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,15 @@ public UnresolvedExpression visitCompareExpr(CompareExprContext ctx) {
String operator = ctx.comparisonOperator().getText();
if ("==".equals(operator)) {
operator = EQUAL.getName().getFunctionName();
} else if ("contains".equalsIgnoreCase(operator)) {

@Swiddis Swiddis Mar 16, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: In principle this should be a const like LIKE but it's probably fine

We already broke the const in the previous if with "==", if someone doesn't like it then they can review this file for consts in another PR.

@ThyTran1402 ThyTran1402 Mar 16, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense to me. Thank you for reviewing it!

UnresolvedExpression left = visit(ctx.left);
UnresolvedExpression right = visit(ctx.right);
if (!(right instanceof Literal) || ((Literal) right).getType() != DataType.STRING) {
throw new SemanticCheckException(
"The right-hand side of 'contains' must be a string literal");
}
String wrapped = "%" + ((Literal) right).getValue() + "%";
return new Compare(ILIKE.getName().getFunctionName(), left, new Literal(wrapped, DataType.STRING));
} else if (LIKE.getName().getFunctionName().equalsIgnoreCase(operator)
&& UnresolvedPlanHelper.isCalciteEnabled(astBuilder.getSettings())) {
operator =
Expand Down
30 changes: 30 additions & 0 deletions ppl/src/test/java/org/opensearch/sql/ppl/parser/AstExpressionBuilderTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@
import org.opensearch.sql.ast.tree.Chart;
import org.opensearch.sql.calcite.plan.OpenSearchConstants;
import org.opensearch.sql.common.antlr.SyntaxCheckException;
import org.opensearch.sql.exception.SemanticCheckException;

public class AstExpressionBuilderTest extends AstBuilderTest {
@Test
Expand Down Expand Up @@ -227,6 +228,35 @@ public void testLikeOperatorCaseInsensitive() {
filter(relation("t"), compare("ilike", field("a"), stringLiteral("pattern"))));
}

@Test
public void testContainsOperatorExpr() {
assertEqual(
"source=t | where a contains 'hello'",
filter(relation("t"), compare("ilike", field("a"), stringLiteral("%hello%"))));

assertEqual(
"source=t | where message contains 'err'",
filter(relation("t"), compare("ilike", field("message"), stringLiteral("%err%"))));
}

@Test
public void testContainsOperatorCaseInsensitive() {
assertEqual(
"source=t | where a CONTAINS 'hello'",
filter(relation("t"), compare("ilike", field("a"), stringLiteral("%hello%"))));

assertEqual(
"source=t | where a Contains 'hello'",
filter(relation("t"), compare("ilike", field("a"), stringLiteral("%hello%"))));
}

@Test
public void testContainsOperatorNonLiteralRhsThrows() {
assertThrows(
SemanticCheckException.class,
() -> assertEqual("source=t | where a contains b", (Node) null));
}

@Test
public void testBooleanIsNullFunction() {
assertEqual(
Expand Down
Loading