Skip to content

Configuring a guard rail has no effect on LLM response. #132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
6 tasks
knihit opened this issue Aug 17, 2024 · 2 comments
Closed
6 tasks

Configuring a guard rail has no effect on LLM response. #132

knihit opened this issue Aug 17, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@knihit
Copy link
Member

knihit commented Aug 17, 2024

Describe the bug
Configure a guardrail when creating a use case. To test, ask questions that the guardrail should filter. use case continues to respond on filtered topics.

To Reproduce
Steps to reproduce the behavior.

Configure a guardrail when creating a use case from the deployment dashboard. The Guardrail was tested from the Bedrock console and works as per the configuration. But when using it through the use case, it does not seem to have any effect. The use case continues to respond on blocked topics.

Expected behavior
A clear and concise description of what you expected to happen.

Please complete the following information about the solution:

  • Version: v2.0.0
  • Region: [e.g. us-east-1]
  • Was the solution modified from the version published on this repository? No
  • If the answer to the previous question was yes, are the changes available on GitHub? No
  • Have you checked your service quotas for the sevices this solution uses? Yes
  • Were there any errors in the CloudWatch Logs? No.

Screenshots
If applicable, add screenshots to help explain your problem (please DO NOT include sensitive information).

Additional context
Add any other context about the problem here.

@knihit knihit added the bug Something isn't working label Aug 17, 2024
@knihit
Copy link
Member Author

knihit commented Aug 17, 2024

We are working on the fix for this issue.

tabdunabi added a commit that referenced this issue Aug 19, 2024
### Changed

-   With the release of [AWS-Solutions-Constructs v2.65.0](https://github.com/awslabs/aws-solutions-constructs/tree/main/source/patterns/%40aws-solutions-constructs/aws-apigatewayv2websocket-sqs), the AWS ApiGateway websocket integration with Amazon SQS Queue is available in the library. Hence the
    implementation has been updated to use this construct.

### Fixed

-   Issue [#131](#131) which caused an issue with rendering non-S3 source URLs from vector store for a RAG based use case.
-   Issue [#132](#132) where configured Guardrails for Amazon Bedrock to have no effect on a use case's text input validations and output responses.
-   Wizard validation failure for SageMaker model selection type, that allowed user to navigate ahead even when the page had failed validations.
-   An AWS WAF rule that blocked larger payloads for HTTP POST request to the `/deployments` endpoint. This restricted configuring large system prompts (over 8 KB) for use case cases.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
tabdunabi added a commit that referenced this issue Aug 19, 2024
### Changed

-   With the release of [AWS-Solutions-Constructs v2.65.0](https://github.com/awslabs/aws-solutions-constructs/tree/main/source/patterns/%40aws-solutions-constructs/aws-apigatewayv2websocket-sqs), the AWS ApiGateway websocket integration with Amazon SQS Queue is available in the library. Hence the implementation has been updated to use this construct.

### Fixed

-   Issue [#131](#131) which caused an issue with rendering non-S3 source URLs from vector store for a RAG based use case.
-   Issue [#132](#132) where configured Guardrails for Amazon Bedrock to have no effect on a use case's text input validations and output responses.
-   Wizard validation failure for SageMaker model selection type, that allowed user to navigate ahead even when the page had failed validations.
-   An AWS WAF rule that blocked larger payloads for HTTP POST request to the `/deployments` endpoint. This restricted configuring large system prompts (over 8 KB) for use case cases.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
@tabdunabi
Copy link
Member

This issue has been resolved in GAAB v2.0.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants