Amazon Bedrock - Add Guardrails dashboard

[agithomas]

• Enhancement

Proposed commit message

Add dashboards to display Amazon Bedrock Guardrails metrics and logs

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• Dashboard verification after upgrade
• Add dataset filter data_stream.dataset : "aws_bedrock.guardrails" or data_stream.dataset : "aws_bedrock.invocation" filter
• Package upgrade test
• Dashboard links validation.

How to test this PR locally

• elastic-package build
• elastic-package stack up -v -d --services package-registry

Related issues

• Relates Amazon Bedrock - Add Guardrails metrics #12250

Screenshots

Added links to the Amazon Bedrock overview dashboard

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[muthu-mps]

Please find my comments in the dashboard image.

• Why the datastream.dataset filter includes guardrails or invocation?

[agithomas]

Thanks @muthu-mps for the inputs.

The intervened Invocations By Policy Type (Input), The intervened Invocations By Policy Type (Output) is related to the Wordcloud visualisation panel. This is why these panels are aligned near the word cloud panel.

These panels don't have much relationship with the Token Unit count (Input) and The token unit count(Output)

[agithomas]

Regarding the feedback on the text - "Text Units utilised by different guardrail policies", we are not only showing the total text unit utilisation by policy type but also the input-output responses, overall sum (without any grouping). These are different dimensions of the same metrics, but related to the guardrail policy.

[agithomas]

Why the datastream.dataset filter includes guardrails or invocation?

The actual request-response involving guardrails is part of the aws_bedrock.invocation ( log-*` dataview) , adding a filter is not possible.

---

Updated the Y-axis label

[muthu-mps]

• Are we okay with the gap in the middle of the overview dashboard?

• Guardrails dashboard:

1. The dashboard filter representation in the overview dashboard and in the guardrails dashboard are different. Can we make this similar in both the places?
2. Invocations with and without guardrail interventions. Name change suggestion: Invocations by (guardrail interventions vs non-guardrail interventions)
3. Guardrail Invocations by Content Source. Update the case for content source to make it consistent with the above chart.
4. Name change suggestion from Number of Invocations where guardrail intervened to Guardrail intervened invocations
5. Text units Utilized - U should be capitalised.

1. Same as point 4 above.
2. Please remove count of text from the name and in the metric label.

[agithomas]

Please find my responses below

1. As mentioned earlier in my previous comment, the same representation is not possible as the filter cannot work on logs-* and metrics-* at the same time.

2. @daniela-elastic, could you please go through the above suggestions (2-5 , 1 & 2) and advice which among of the above comments is to be considered?

[muthu-mps]

1. As mentioned earlier in my previous comment, the same representation is not possible as the filter cannot work on logs-* and metrics-* at the same time.

Okay, You mean the UI doesn't allow to save the filter with a custom name if we create more than one filter.

[agithomas]

Okay, You mean the UI doesn't allow to save the filter with a custom name if we create more than one filter.

multi-filters are supported provided the data view is same. But, in this case, we are using two different dataviews - logs-* and metrics-*.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: b021e333f155854e69eefd61224307bbff86d1da

History

• 💚 Build #20513 succeeded 7766be639b194e1ac4590ccf23a935d6c9c1111d
• 💚 Build #20492 succeeded 5b87b9799b7a94e88b3465d8038cf5a5f02d44f1
• 💔 Build #20477 failed d1efc98b2d942e22d8cb432a76e4fdf5e43dcfda
• 💚 Build #20354 succeeded e2dede5876bdd610d5b5c001ff434ad2cd7d6519
• 💚 Build #20341 succeeded 4fa584a6409ba2cb4a7be2062cfc2723a59ca886

cc @agithomas

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[muthu-mps]

LGTM!

[elastic-vault-github-plugin-prod]

Package aws_bedrock - 0.21.0 containing this change is available at https://epr.elastic.co/package/aws_bedrock/0.21.0/

[daniela-elastic]

Agi please see comments:

1. "Total Invocations" should be "Total invocations" (small i). See screenshot below

2. Also, can we consider making the widgets for total invocations, total tokens and total errors slightly narrower, that way there's not too much space between the title of the widget and the number. Alternatively, right justify the number so the beginning of the number is justified with the beginning of the widget name

3. General comment. use small caps in widget names (unless any of the words is a product/service name eg Guardrails; however small caps if the word is a noun that is not the service name, eg guardrail which describes qualitatively what is invoked). See screenshot below

4. no.1 use "aws_bedrock_guardrails" (the first one), see screenshot below

5. no.2 change to small caps - "Invocations - with and without guardrail interventions", see screenshot below

6. no.3 change to small caps - "Guardrail invocations by content source (Input & Output)", see screenshot below

7. no.4 change to small caps - "Number of invocations where guardrails intervened", see screenshot below

8. no.5 change to small caps and use singular when a noun is used as an adjective - "Text unit utilization". For the y-axis - "Text units"

9. Change y-axis to small caps w/ exception of Input/Output - "Guardrail API invocations (Input) / (Output)"

10. Change to "Text unit utilization"

11. Change remaining widget names to small caps like above, eg "Invocation latency" instead of "Invocation Latency"

[daniela-elastic]

12. no.1 Change to small caps - "Number of invocations where the guardrails intervened - by policy type (Input)

13. no.2 change to "Interventions" - in both widgets

14. no.2 change to "Text units - by policy type (Input) / (Output)" - in both places accordingly

15. At top of the section, change "Guardrail policy types of intervened invocations" to "Guardrail policy types for interventions"