Skip to content

When updating a use case, setting the Score threshold to DISABLED does not seem to take effect. #154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
6 tasks done
hiraken-w opened this issue Oct 5, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@hiraken-w
Copy link

hiraken-w commented Oct 5, 2024

Describe the bug

As per the following documentation, we can configure Kendra's Advanced RAG configurations. However, once we set the Score threshold to a value other than DISABLED, like LOW, we cannot revert it back to DISABLED in subsequent updates. On the web app, there is no error shown, and it appears to be successful, but the change is not reflected.

https://docs.aws.amazon.com/solutions/latest/generative-ai-application-builder-on-aws/advanced-knowledge-base-settings.html

If we update to set the Score threshold to DISABLED, the request body for the PATCH method to API Gateway does not include the ScoreThreshold parameter. In this case, it is considered necessary to perform processing such as deleting the item keys (ScoreThreshold, StorageCapacityUnits) from the DynamoDB table.

To Reproduce

  1. Create a Generative AI Application Builder.
  2. Deploy a new use case. Leave the default settings as is. Select Kendra for the Knowledge base type, and set a value other than DISABLED for the Score Threshold in Advanced RAG configurations.
  3. Update the use case. Set DISABLED for the Score Threshold in Advanced RAG configurations.

As a result, the Score Threshold does not become DISABLED.

Expected behavior

The Score Threshold value in Advanced RAG configurations can be updated to DISABLED.

Please complete the following information about the solution:

  • Version: v2.0.4
  • Region: us-east-1
  • Was the solution modified from the version published on this repository?: No changes have been made.
  • If the answer to the previous question was yes, are the changes available on GitHub?
  • Have you checked your service quotas for the sevices this solution uses?: This is not a service quota issue.
  • Were there any errors in the CloudWatch Logs?: The Lambda function's create_config_string execution was successful.
@hiraken-w hiraken-w added the bug Something isn't working label Oct 5, 2024
@ihmaws
Copy link
Member

ihmaws commented Oct 6, 2024

Thank you @hiraken-w for the detailed bug report!

I was able to reproduce the issue and identify the root cause. The team will have a look at a fix and include a patch in our next release.

We'll let you know here once a fix is available. Thanks again!

@jamesnixon-aws
Copy link
Contributor

Issue resolved in release v2.1.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants