Skip to content

[Response Ops][Reporting] Add health API to inform whether conditions are sufficient for scheduled reports#216857

Merged
ymao1 merged 3 commits intoelastic:mainfrom
ymao1:reporting-health-api
Apr 7, 2025
Merged

[Response Ops][Reporting] Add health API to inform whether conditions are sufficient for scheduled reports#216857
ymao1 merged 3 commits intoelastic:mainfrom
ymao1:reporting-health-api

Conversation

@ymao1
Copy link
Contributor

@ymao1 ymao1 commented Apr 2, 2025

Resolves #216319

Summary

Adds an internal reporting health API to return whether conditions are sufficient to support scheduled reports. For scheduled reporting, we need for security and API keys to be enabled in Elasticsearch and for a permanent encryption key to be set for the encrypted saved objects plugin.

GET kbn:/internal/reporting/_health

Response 
{
  "has_permanent_encryption_key": true,
  "is_sufficiently_secure": true
}

The issue also mentions returning whether a preconfigured email service is configured, but that will be done as part of the main scheduled reporting task.

To Verify

  1. Run kibana and ES with no special flags, both flags should be true
  2. Run ES with -E xpack.security.enabled=false. is_sufficiently_secure should be set to false
  3. Run ES With -E xpack.security.authc.api_key.enabled=false. is_sufficient_secure should be set to false

Note that in dev mode, an encryption key is auto-set if not defined in the Kibana yml so has_permanent_encryption_key will always return true in dev mode.

@ymao1 ymao1 force-pushed the reporting-health-api branch from c59f09e to 9dc3bc3 Compare April 2, 2025 17:55
@ymao1 ymao1 changed the title wip [Response Ops][Reporting] Add health API to inform whether conditions are sufficient for scheduled reports Apr 2, 2025
@ymao1 ymao1 force-pushed the reporting-health-api branch from 9dc3bc3 to c6ad2a4 Compare April 2, 2025 18:05
@ymao1 ymao1 force-pushed the reporting-health-api branch from c6ad2a4 to 926fa52 Compare April 2, 2025 18:07
@ymao1 ymao1 self-assigned this Apr 2, 2025
@ymao1 ymao1 added release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// Feature:Reporting:Framework Reporting issues pertaining to the overall framework backport:version Backport to applied version labels v9.1.0 v8.19.0 labels Apr 2, 2025
@ymao1 ymao1 marked this pull request as ready for review April 2, 2025 21:19
@ymao1 ymao1 requested review from a team as code owners April 2, 2025 21:19
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@ymao1
Copy link
Contributor Author

ymao1 commented Apr 7, 2025

@elasticmachine merge upstream

@elasticmachine elasticmachine requested a review from a team as a code owner April 7, 2025 13:01
@elasticmachine
Copy link
Contributor

elasticmachine commented Apr 7, 2025

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #1 / should filter the table correctly
  • [job] [logs] Jest Tests #1 / should filter the table correctly

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/reporting-common 88 89 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
fleet 1.7MB 1.7MB +20.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
reporting 48.9KB 48.9KB +20.0B
Unknown metric groups

API count

id before after diff
@kbn/reporting-common 98 99 +1

History

cc @ymao1

Copy link
Contributor

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, works as advertised!

@ymao1 ymao1 merged commit 3d54923 into elastic:main Apr 7, 2025
9 checks passed
@ymao1 ymao1 deleted the reporting-health-api branch April 7, 2025 20:46
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/14319138149

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.x Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 216857

Questions ?

Please refer to the Backport tool documentation

@ymao1
Copy link
Contributor Author

ymao1 commented Apr 7, 2025

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

ymao1 added a commit to ymao1/kibana that referenced this pull request Apr 7, 2025
… are sufficient for scheduled reports (elastic#216857)

Resolves elastic#216319

## Summary

Adds an internal reporting health API to return whether conditions are
sufficient to support scheduled reports. For scheduled reporting, we
need for security and API keys to be enabled in Elasticsearch and for a
permanent encryption key to be set for the encrypted saved objects
plugin.

```
GET kbn:/internal/reporting/_health

Response
{
  "has_permanent_encryption_key": true,
  "is_sufficiently_secure": true
}
```

The issue also mentions returning whether a preconfigured email service
is configured, but that will be done as part of the main scheduled
reporting task.

## To Verify

1. Run kibana and ES with no special flags, both flags should be `true`
2. Run ES with `-E xpack.security.enabled=false`.
`is_sufficiently_secure` should be set to `false`
3. Run ES With `-E xpack.security.authc.api_key.enabled=false`.
`is_sufficient_secure` should be set to `false`

Note that in dev mode, an encryption key is auto-set if not defined in
the Kibana yml so `has_permanent_encryption_key` will always return
`true` in dev mode.

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
(cherry picked from commit 3d54923)

# Conflicts:
#	src/platform/packages/private/kbn-reporting/common/routes.ts
#	x-pack/platform/plugins/private/reporting/server/core.ts
#	x-pack/platform/plugins/private/reporting/server/routes/index.ts
ymao1 added a commit that referenced this pull request Apr 8, 2025
…itions are sufficient for scheduled reports (#216857) (#217412)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Response Ops][Reporting] Add health API to inform whether conditions
are sufficient for scheduled reports
(#216857)](#216857)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Ying
Mao","email":"ying.mao@elastic.co"},"sourceCommit":{"committedDate":"2025-04-07T20:46:02Z","message":"[Response
Ops][Reporting] Add health API to inform whether conditions are
sufficient for scheduled reports (#216857)\n\nResolves
https://github.com/elastic/kibana/issues/216319\n\n## Summary\n\nAdds an
internal reporting health API to return whether conditions
are\nsufficient to support scheduled reports. For scheduled reporting,
we\nneed for security and API keys to be enabled in Elasticsearch and
for a\npermanent encryption key to be set for the encrypted saved
objects\nplugin.\n\n```\nGET kbn:/internal/reporting/_health\n\nResponse
\n{\n \"has_permanent_encryption_key\": true,\n
\"is_sufficiently_secure\": true\n}\n```\n\nThe issue also mentions
returning whether a preconfigured email service\nis configured, but that
will be done as part of the main scheduled\nreporting task.\n\n## To
Verify\n\n1. Run kibana and ES with no special flags, both flags should
be `true`\n2. Run ES with `-E
xpack.security.enabled=false`.\n`is_sufficiently_secure` should be set
to `false`\n3. Run ES With `-E
xpack.security.authc.api_key.enabled=false`.\n`is_sufficient_secure`
should be set to `false`\n\nNote that in dev mode, an encryption key is
auto-set if not defined in\nthe Kibana yml so
`has_permanent_encryption_key` will always return\n`true` in dev
mode.\n\n---------\n\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"3d54923123b37ff0c1d7e51067ac30a20965461b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:ResponseOps","Feature:Reporting:Framework","backport:version","v9.1.0","v8.19.0"],"title":"[Response
Ops][Reporting] Add health API to inform whether conditions are
sufficient for scheduled
reports","number":216857,"url":"https://github.com/elastic/kibana/pull/216857","mergeCommit":{"message":"[Response
Ops][Reporting] Add health API to inform whether conditions are
sufficient for scheduled reports (#216857)\n\nResolves
https://github.com/elastic/kibana/issues/216319\n\n## Summary\n\nAdds an
internal reporting health API to return whether conditions
are\nsufficient to support scheduled reports. For scheduled reporting,
we\nneed for security and API keys to be enabled in Elasticsearch and
for a\npermanent encryption key to be set for the encrypted saved
objects\nplugin.\n\n```\nGET kbn:/internal/reporting/_health\n\nResponse
\n{\n \"has_permanent_encryption_key\": true,\n
\"is_sufficiently_secure\": true\n}\n```\n\nThe issue also mentions
returning whether a preconfigured email service\nis configured, but that
will be done as part of the main scheduled\nreporting task.\n\n## To
Verify\n\n1. Run kibana and ES with no special flags, both flags should
be `true`\n2. Run ES with `-E
xpack.security.enabled=false`.\n`is_sufficiently_secure` should be set
to `false`\n3. Run ES With `-E
xpack.security.authc.api_key.enabled=false`.\n`is_sufficient_secure`
should be set to `false`\n\nNote that in dev mode, an encryption key is
auto-set if not defined in\nthe Kibana yml so
`has_permanent_encryption_key` will always return\n`true` in dev
mode.\n\n---------\n\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"3d54923123b37ff0c1d7e51067ac30a20965461b"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/216857","number":216857,"mergeCommit":{"message":"[Response
Ops][Reporting] Add health API to inform whether conditions are
sufficient for scheduled reports (#216857)\n\nResolves
https://github.com/elastic/kibana/issues/216319\n\n## Summary\n\nAdds an
internal reporting health API to return whether conditions
are\nsufficient to support scheduled reports. For scheduled reporting,
we\nneed for security and API keys to be enabled in Elasticsearch and
for a\npermanent encryption key to be set for the encrypted saved
objects\nplugin.\n\n```\nGET kbn:/internal/reporting/_health\n\nResponse
\n{\n \"has_permanent_encryption_key\": true,\n
\"is_sufficiently_secure\": true\n}\n```\n\nThe issue also mentions
returning whether a preconfigured email service\nis configured, but that
will be done as part of the main scheduled\nreporting task.\n\n## To
Verify\n\n1. Run kibana and ES with no special flags, both flags should
be `true`\n2. Run ES with `-E
xpack.security.enabled=false`.\n`is_sufficiently_secure` should be set
to `false`\n3. Run ES With `-E
xpack.security.authc.api_key.enabled=false`.\n`is_sufficient_secure`
should be set to `false`\n\nNote that in dev mode, an encryption key is
auto-set if not defined in\nthe Kibana yml so
`has_permanent_encryption_key` will always return\n`true` in dev
mode.\n\n---------\n\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"3d54923123b37ff0c1d7e51067ac30a20965461b"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
ymao1 added a commit that referenced this pull request May 8, 2025
…ication email service is available (#219785)

## Summary

Updates the internal reporting health API from
#216857 to return whether the
notification email service is available. We'll be using this
notification email service to send the scheduled report notifications so
this lets us know if that service is available.

```
GET kbn:/internal/reporting/_health

Response 
{
  "has_permanent_encryption_key": true,
  "is_sufficiently_secure": true,
  "are_notifications_enabled": true
}
```

## To Verify
1. Run kibana and ES with a preconfigured email connector in the
`kibana.yml` and the notification configured to use the connector

```
notifications.connectors.default.email: gmail
xpack.actions.preconfigured:
  gmail:
    name: 'email: my gmail'
    actionTypeId: '.email'
    config:
      <config>
    secrets:
      <secrets>
```
2. Access the health API, `are_notifications_enabled` should be `true`
3. Remove the config and restart Kibana
4. Access the health API, `are_notifications_enabled` should be `false`

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
ymao1 added a commit to ymao1/kibana that referenced this pull request May 8, 2025
…ication email service is available (elastic#219785)

## Summary

Updates the internal reporting health API from
elastic#216857 to return whether the
notification email service is available. We'll be using this
notification email service to send the scheduled report notifications so
this lets us know if that service is available.

```
GET kbn:/internal/reporting/_health

Response
{
  "has_permanent_encryption_key": true,
  "is_sufficiently_secure": true,
  "are_notifications_enabled": true
}
```

## To Verify
1. Run kibana and ES with a preconfigured email connector in the
`kibana.yml` and the notification configured to use the connector

```
notifications.connectors.default.email: gmail
xpack.actions.preconfigured:
  gmail:
    name: 'email: my gmail'
    actionTypeId: '.email'
    config:
      <config>
    secrets:
      <secrets>
```
2. Access the health API, `are_notifications_enabled` should be `true`
3. Remove the config and restart Kibana
4. Access the health API, `are_notifications_enabled` should be `false`

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
(cherry picked from commit 25a873c)
ymao1 added a commit that referenced this pull request May 8, 2025
…r notification email service is available (#219785) (#220555)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[Response Ops][Reporting] Updating health API to return whether
notification email service is available
(#219785)](#219785)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Ying
Mao","email":"ying.mao@elastic.co"},"sourceCommit":{"committedDate":"2025-05-08T14:11:53Z","message":"[Response
Ops][Reporting] Updating health API to return whether notification email
service is available (#219785)\n\n## Summary\n\nUpdates the internal
reporting health API from\nhttps://github.com//pull/216857
to return whether the\nnotification email service is available. We'll be
using this\nnotification email service to send the scheduled report
notifications so\nthis lets us know if that service is
available.\n\n```\nGET kbn:/internal/reporting/_health\n\nResponse \n{\n
\"has_permanent_encryption_key\": true,\n \"is_sufficiently_secure\":
true,\n \"are_notifications_enabled\": true\n}\n```\n\n## To Verify\n1.
Run kibana and ES with a preconfigured email connector in
the\n`kibana.yml` and the notification configured to use the
connector\n\n```\nnotifications.connectors.default.email:
gmail\nxpack.actions.preconfigured:\n gmail:\n name: 'email: my gmail'\n
actionTypeId: '.email'\n config:\n <config>\n secrets:\n
<secrets>\n```\n2. Access the health API, `are_notifications_enabled`
should be `true`\n3. Remove the config and restart Kibana\n4. Access the
health API, `are_notifications_enabled` should be
`false`\n\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"25a873c86c2c7fe1f5b5f345d2ee349f721c7c2a","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:ResponseOps","Feature:Reporting:Framework","backport:version","v9.1.0","v8.19.0"],"title":"[Response
Ops][Reporting] Updating health API to return whether notification email
service is
available","number":219785,"url":"https://github.com/elastic/kibana/pull/219785","mergeCommit":{"message":"[Response
Ops][Reporting] Updating health API to return whether notification email
service is available (#219785)\n\n## Summary\n\nUpdates the internal
reporting health API from\nhttps://github.com//pull/216857
to return whether the\nnotification email service is available. We'll be
using this\nnotification email service to send the scheduled report
notifications so\nthis lets us know if that service is
available.\n\n```\nGET kbn:/internal/reporting/_health\n\nResponse \n{\n
\"has_permanent_encryption_key\": true,\n \"is_sufficiently_secure\":
true,\n \"are_notifications_enabled\": true\n}\n```\n\n## To Verify\n1.
Run kibana and ES with a preconfigured email connector in
the\n`kibana.yml` and the notification configured to use the
connector\n\n```\nnotifications.connectors.default.email:
gmail\nxpack.actions.preconfigured:\n gmail:\n name: 'email: my gmail'\n
actionTypeId: '.email'\n config:\n <config>\n secrets:\n
<secrets>\n```\n2. Access the health API, `are_notifications_enabled`
should be `true`\n3. Remove the config and restart Kibana\n4. Access the
health API, `are_notifications_enabled` should be
`false`\n\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"25a873c86c2c7fe1f5b5f345d2ee349f721c7c2a"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/219785","number":219785,"mergeCommit":{"message":"[Response
Ops][Reporting] Updating health API to return whether notification email
service is available (#219785)\n\n## Summary\n\nUpdates the internal
reporting health API from\nhttps://github.com//pull/216857
to return whether the\nnotification email service is available. We'll be
using this\nnotification email service to send the scheduled report
notifications so\nthis lets us know if that service is
available.\n\n```\nGET kbn:/internal/reporting/_health\n\nResponse \n{\n
\"has_permanent_encryption_key\": true,\n \"is_sufficiently_secure\":
true,\n \"are_notifications_enabled\": true\n}\n```\n\n## To Verify\n1.
Run kibana and ES with a preconfigured email connector in
the\n`kibana.yml` and the notification configured to use the
connector\n\n```\nnotifications.connectors.default.email:
gmail\nxpack.actions.preconfigured:\n gmail:\n name: 'email: my gmail'\n
actionTypeId: '.email'\n config:\n <config>\n secrets:\n
<secrets>\n```\n2. Access the health API, `are_notifications_enabled`
should be `true`\n3. Remove the config and restart Kibana\n4. Access the
health API, `are_notifications_enabled` should be
`false`\n\nCo-authored-by: Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"25a873c86c2c7fe1f5b5f345d2ee349f721c7c2a"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request May 29, 2025
…ication email service is available (elastic#219785)

## Summary

Updates the internal reporting health API from
elastic#216857 to return whether the
notification email service is available. We'll be using this
notification email service to send the scheduled report notifications so
this lets us know if that service is available.

```
GET kbn:/internal/reporting/_health

Response 
{
  "has_permanent_encryption_key": true,
  "is_sufficiently_secure": true,
  "are_notifications_enabled": true
}
```

## To Verify
1. Run kibana and ES with a preconfigured email connector in the
`kibana.yml` and the notification configured to use the connector

```
notifications.connectors.default.email: gmail
xpack.actions.preconfigured:
  gmail:
    name: 'email: my gmail'
    actionTypeId: '.email'
    config:
      <config>
    secrets:
      <secrets>
```
2. Access the health API, `are_notifications_enabled` should be `true`
3. Remove the config and restart Kibana
4. Access the health API, `are_notifications_enabled` should be `false`

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
qn895 pushed a commit to qn895/kibana that referenced this pull request Jun 3, 2025
…ication email service is available (elastic#219785)

## Summary

Updates the internal reporting health API from
elastic#216857 to return whether the
notification email service is available. We'll be using this
notification email service to send the scheduled report notifications so
this lets us know if that service is available.

```
GET kbn:/internal/reporting/_health

Response 
{
  "has_permanent_encryption_key": true,
  "is_sufficiently_secure": true,
  "are_notifications_enabled": true
}
```

## To Verify
1. Run kibana and ES with a preconfigured email connector in the
`kibana.yml` and the notification configured to use the connector

```
notifications.connectors.default.email: gmail
xpack.actions.preconfigured:
  gmail:
    name: 'email: my gmail'
    actionTypeId: '.email'
    config:
      <config>
    secrets:
      <secrets>
```
2. Access the health API, `are_notifications_enabled` should be `true`
3. Remove the config and restart Kibana
4. Access the health API, `are_notifications_enabled` should be `false`

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels Feature:Reporting:Framework Reporting issues pertaining to the overall framework release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// v8.19.0 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Response Ops][Reporting] Reporting health API

5 participants