Skip to content

Turn on internal API restriction for serverless tests #162636

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jloleysens merged 9 commits into from
Aug 1, 2023
Merged

Turn on internal API restriction for serverless tests #162636

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
6b8403b
added restrict internal APIs flag to serverless base config
jloleysens Jul 27, 2023
6fc50c1
update security tests
jloleysens Jul 27, 2023
301afab
resolve merge conflicts
jloleysens Jul 28, 2023
e58a38e
updated o11y calls
jloleysens Jul 28, 2023
d30a346
Merge branch 'main' into serverless-e2e-test
jloleysens Jul 28, 2023
f74cab2
more internal
jloleysens Jul 28, 2023
c4310cc
set internal origin headers on APM client
jloleysens Jul 28, 2023
783482d
remove duplicate setting of port in base config
jloleysens Jul 28, 2023
ad006af
another missin header
jloleysens Jul 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/kbn-test/src/kbn_client/kbn_client_requester.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ export class KbnClientRequester {
headers: {
...options.headers,
'kbn-xsrf': 'kbn-client',
'x-elastic-internal-origin': 'kbn-client',
},
httpsAgent: this.httpsAgent,
responseType: options.responseType,
Expand Down
1 change: 1 addition & 0 deletions x-pack/test/api_integration/services/usage_api.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ export function UsageAPIProvider({ getService }: FtrProviderContext) {
const { body } = await supertest
.post('/api/telemetry/v2/clusters/_stats')
.set('kbn-xsrf', 'xxx')
.set('x-elastic-internal-origin', 'xxx')
.send({ refreshCache: true, ...payload })
.expect(200);
return body;
Expand Down
5 changes: 4 additions & 1 deletion x-pack/test/apm_api_integration/common/apm_api_supertest.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,10 @@ export function createApmApiClient(st: supertest.SuperTest<supertest.Test>) {
const { method, pathname, version } = formatRequest(endpoint, params.path);
const url = format({ pathname, query: params?.query });

const headers: Record<string, string> = { 'kbn-xsrf': 'foo' };
const headers: Record<string, string> = {
'kbn-xsrf': 'foo',
'x-elastic-internal-origin': 'foo',
};

if (version) {
headers['Elastic-Api-Version'] = version;
Expand Down
2 changes: 1 addition & 1 deletion x-pack/test_serverless/api_integration/test_suites/common/security_response_headers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ export default function ({ getService }: FtrProviderContext) {
it('redirect endpoint response contains default security headers', async () => {
const { header } = await supertest
.get(`/logout`)
.set(svlCommonApi.getCommonRequestHeader())
.set(svlCommonApi.getInternalRequestHeader())
.expect(200);

expect(header).toBeDefined();
Expand Down
2 changes: 1 addition & 1 deletion x-pack/test_serverless/api_integration/test_suites/common/security_users.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ export default function ({ getService }: FtrProviderContext) {
it('rejects request to create user', async () => {
const { body, status } = await supertest
.post(`/internal/security/users/some_testuser`)
.set(svlCommonApi.getCommonRequestHeader())
.set(svlCommonApi.getInternalRequestHeader())
.send({ username: 'some_testuser', password: 'testpassword', roles: [] });

// in a non-serverless environment this would succeed with a 200
Expand Down
5 changes: 4 additions & 1 deletion ...api_integration/test_suites/observability/apm_api_integration/common/apm_api_supertest.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,10 @@ export function createApmApiClient(st: supertest.SuperTest<supertest.Test>) {
const { method, pathname, version } = formatRequest(endpoint, params.path);
const url = format({ pathname, query: params?.query });

const headers: Record<string, string> = { 'kbn-xsrf': 'foo' };
const headers: Record<string, string> = {
'kbn-xsrf': 'foo',
'x-elastic-internal-origin': 'foo',
};

if (version) {
headers['Elastic-Api-Version'] = version;
Expand Down
4 changes: 2 additions & 2 deletions x-pack/test_serverless/api_integration/test_suites/observability/fleet.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ export default function ({ getService }: FtrProviderContext) {
it('rejects request to create a new fleet server hosts', async () => {
const { body, status } = await supertest
.post('/api/fleet/fleet_server_hosts')
.set(svlCommonApi.getCommonRequestHeader())
.set(svlCommonApi.getInternalRequestHeader())
Copy link
Copy Markdown
Contributor

@TinaHeiligers TinaHeiligers Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the API's public, not internal. Fleet has an open issue to change to 'public' again: https://github.com/elastic/ingest-dev/issues/1921

Copy link
Copy Markdown
Contributor

@achyutjhunjhunwala achyutjhunjhunwala Jul 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means until the linked issue is resolved, this API would be treated as internal ?

.send({
name: 'test',
host_urls: ['https://localhost:8220'],
Expand All @@ -34,7 +34,7 @@ export default function ({ getService }: FtrProviderContext) {
it('rejects request to create a new proxy', async () => {
const { body, status } = await supertest
.post('/api/fleet/proxies')
.set(svlCommonApi.getCommonRequestHeader())
.set(svlCommonApi.getInternalRequestHeader())
Copy link
Copy Markdown
Contributor

@TinaHeiligers TinaHeiligers Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

public

.send({
name: 'test',
url: 'https://localhost:8220',
Expand Down
2 changes: 2 additions & 0 deletions .../test_serverless/api_integration/test_suites/observability/helpers/alerting_api_helper.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ export async function createIndexConnector({
const { body } = await supertest
.post(`/api/actions/connector`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo')
.send({
name,
config: {
Expand Down Expand Up @@ -54,6 +55,7 @@ export async function createRule({
const { body } = await supertest
.post(`/api/alerting/rule`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo')
Copy link
Copy Markdown
Contributor

@TinaHeiligers TinaHeiligers Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 . confirmed with @elastic/actionable-observability, their APIs are 'internal'

.send({
params,
consumer,
Expand Down
5 changes: 4 additions & 1 deletion ...serverless/api_integration/test_suites/observability/helpers/alerting_wait_for_helpers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,10 @@ export async function waitForRuleStatus({
}): Promise<Record<string, any>> {
return pRetry(
async () => {
const response = await supertest.get(`/api/alerting/rule/${id}`);
const response = await supertest
.get(`/api/alerting/rule/${id}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
Copy link
Copy Markdown
Contributor

@TinaHeiligers TinaHeiligers Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

const { execution_status: executionStatus } = response.body || {};
const { status } = executionStatus || {};
if (status !== expectedStatus) {
Expand Down
2 changes: 2 additions & 0 deletions x-pack/test_serverless/api_integration/test_suites/observability/helpers/data_view.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ export const createDataView = async ({
const { body } = await supertest
.post(`/api/content_management/rpc/create`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo')
Copy link
Copy Markdown
Contributor

@TinaHeiligers TinaHeiligers Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

.send({
contentTypeId: 'index-pattern',
data: {
Expand Down Expand Up @@ -49,6 +50,7 @@ export const deleteDataView = async ({
const { body } = await supertest
.post(`/api/content_management/rpc/delete`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo')
.send({
contentTypeId: 'index-pattern',
id,
Expand Down
10 changes: 8 additions & 2 deletions ...test_serverless/api_integration/test_suites/observability/threshold_rule/avg_pct_fired.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,14 @@ export default function ({ getService }: FtrProviderContext) {
});

after(async () => {
await supertest.delete(`/api/alerting/rule/${ruleId}`).set('kbn-xsrf', 'foo');
await supertest.delete(`/api/actions/connector/${actionId}`).set('kbn-xsrf', 'foo');
await supertest
.delete(`/api/alerting/rule/${ruleId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
Copy link
Copy Markdown
Contributor

@TinaHeiligers TinaHeiligers Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

await supertest
.delete(`/api/actions/connector/${actionId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
Copy link
Copy Markdown
Contributor

@TinaHeiligers TinaHeiligers Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

await esClient.deleteByQuery({
index: THRESHOLD_RULE_ALERT_INDEX,
query: { term: { 'kibana.alert.rule.uuid': ruleId } },
Expand Down
10 changes: 8 additions & 2 deletions ...st_serverless/api_integration/test_suites/observability/threshold_rule/avg_pct_no_data.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,14 @@ export default function ({ getService }: FtrProviderContext) {
});

after(async () => {
await supertest.delete(`/api/alerting/rule/${ruleId}`).set('kbn-xsrf', 'foo');
await supertest.delete(`/api/actions/connector/${actionId}`).set('kbn-xsrf', 'foo');
await supertest
.delete(`/api/alerting/rule/${ruleId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
await supertest
.delete(`/api/actions/connector/${actionId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
await esClient.deleteByQuery({
index: THRESHOLD_RULE_ALERT_INDEX,
query: { term: { 'kibana.alert.rule.uuid': ruleId } },
Expand Down
10 changes: 8 additions & 2 deletions ...ess/api_integration/test_suites/observability/threshold_rule/custom_eq_avg_bytes_fired.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,14 @@ export default function ({ getService }: FtrProviderContext) {
});

after(async () => {
await supertest.delete(`/api/alerting/rule/${ruleId}`).set('kbn-xsrf', 'foo');
await supertest.delete(`/api/actions/connector/${actionId}`).set('kbn-xsrf', 'foo');
await supertest
.delete(`/api/alerting/rule/${ruleId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
await supertest
.delete(`/api/actions/connector/${actionId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
await esClient.deleteByQuery({
index: THRESHOLD_RULE_ALERT_INDEX,
query: { term: { 'kibana.alert.rule.uuid': ruleId } },
Expand Down
10 changes: 8 additions & 2 deletions ...verless/api_integration/test_suites/observability/threshold_rule/documents_count_fired.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,14 @@ export default function ({ getService }: FtrProviderContext) {
});

after(async () => {
await supertest.delete(`/api/alerting/rule/${ruleId}`).set('kbn-xsrf', 'foo');
await supertest.delete(`/api/actions/connector/${actionId}`).set('kbn-xsrf', 'foo');
await supertest
.delete(`/api/alerting/rule/${ruleId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
await supertest
.delete(`/api/actions/connector/${actionId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
await esClient.deleteByQuery({
index: THRESHOLD_RULE_ALERT_INDEX,
query: { term: { 'kibana.alert.rule.uuid': ruleId } },
Expand Down
10 changes: 8 additions & 2 deletions ...est_serverless/api_integration/test_suites/observability/threshold_rule/group_by_fired.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,14 @@ export default function ({ getService }: FtrProviderContext) {
});

after(async () => {
await supertest.delete(`/api/alerting/rule/${ruleId}`).set('kbn-xsrf', 'foo');
await supertest.delete(`/api/actions/connector/${actionId}`).set('kbn-xsrf', 'foo');
await supertest
.delete(`/api/alerting/rule/${ruleId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
await supertest
.delete(`/api/actions/connector/${actionId}`)
.set('kbn-xsrf', 'foo')
.set('x-elastic-internal-origin', 'foo');
await esClient.deleteByQuery({
index: THRESHOLD_RULE_ALERT_INDEX,
query: { term: { 'kibana.alert.rule.uuid': ruleId } },
Expand Down
4 changes: 2 additions & 2 deletions x-pack/test_serverless/api_integration/test_suites/security/fleet.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ export default function ({ getService }: FtrProviderContext) {
it('rejects request to create a new fleet server hosts', async () => {
const { body, status } = await supertest
.post('/api/fleet/fleet_server_hosts')
.set(svlCommonApi.getCommonRequestHeader())
.set(svlCommonApi.getInternalRequestHeader())
.send({
name: 'test',
host_urls: ['https://localhost:8220'],
Expand All @@ -34,7 +34,7 @@ export default function ({ getService }: FtrProviderContext) {
it('rejects request to create a new proxy', async () => {
const { body, status } = await supertest
.post('/api/fleet/proxies')
.set(svlCommonApi.getCommonRequestHeader())
.set(svlCommonApi.getInternalRequestHeader())
.send({
name: 'test',
url: 'https://localhost:8220',
Expand Down
1 change: 1 addition & 0 deletions x-pack/test_serverless/shared/config.base.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ export default async () => {
},
sourceArgs: ['--no-base-path', '--env.name=development'],
serverArgs: [
`--server.restrictInternalApis=true`,
Copy link
Copy Markdown
Contributor Author

@jloleysens jloleysens Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most important change of the PR

`--server.port=${kbnTestConfig.getPort()}`,
'--status.allowAnonymous=true',
// We shouldn't embed credentials into the URL since Kibana requests to Elasticsearch should
Expand Down