Enable PPL rest command endpoints for sql 3.8.0 integ-test cluster - #6299
Conversation
The PPL `rest` command (opensearch-project/sql#5599) is disabled by default via an empty, node-scoped allow-list (plugins.ppl.rest.allowed_endpoints). The sql repo's own integ-test clusters opt into the registered endpoints, but the 3.8.0 distribution test manifest was not updated, so the release integ-test cluster boots with the command disabled. Every '| rest ...' test in CalcitePPLRestIT, CalciteExplainIT, NewAddedCommandsIT and the CalciteNoPushdownIT suite then fails with 'the rest command is disabled on this cluster' (opensearch-project/sql#5620). Add the same endpoint allow-list the sql repo uses so the release integ-test cluster exercises the enabled path. Signed-off-by: Kai Huang <ahkcs@amazon.com>
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
@ahkcs I believe it would need some code changes on python level as well. AFAIK |
It looks like a valid key-pair with list values. |
Yes I misread the error. My bad. Looks like re-run is fixing it |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6299 +/- ##
=======================================
Coverage 96.68% 96.68%
=======================================
Files 407 407
Lines 19443 19443
=======================================
Hits 18799 18799
Misses 644 644 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Description
The PPL
restcommand (opensearch-project/sql#5599) is disabled by default: it is gated behindplugins.ppl.rest.allowed_endpoints, a node-scoped setting whose default is an empty allow-list. Because the setting isNodeScope(non-dynamic), it must be provided at cluster startup.The sql repo's own
integ-test/build.gradleopts its test clusters into the registered endpoints, which is why the tests pass in the sql repo's CI. But the 3.8.0 distribution test manifest was never updated to match, so the release integ-test cluster boots with the command disabled.As a result, every
| rest ...integration test fails on the distribution cluster withthe rest command is disabled on this cluster— across bothwith-securityandwithout-security, on every platform/arch/dist. Affected classes:org.opensearch.sql.calcite.remote.CalcitePPLRestITorg.opensearch.sql.calcite.remote.CalciteExplainIT(explainRestCommand)org.opensearch.sql.calcite.remote.CalciteNewAddedCommandsIT(testRest)org.opensearch.sql.calcite.CalciteNoPushdownIT(suite runner aggregating the above)This adds the same endpoint allow-list the sql repo enables so the release integ-test cluster exercises the enabled path. The value is rendered into
opensearch.ymlas a native YAML list (matching howplugins.destination.host.deny_listandpath.repoare already configured for other components in this manifest), which is the correct representation for an OpenSearchlistSetting.Only the
3.8.0manifest is affected — therestcommand first ships in 3.8.0, so earlier release trains (3.7.x, 3.6.x) do not have the feature and need no change.Issues Resolved
Fixes the current failure mode in opensearch-project/sql#5620 (build 12088).
Testing
TestManifestloader (Cerberus schema): loads and validates OK.yaml.dumpof the config rendersplugins.ppl.rest.allowed_endpointsas a native YAML list in the generatedopensearch.yml.Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.