Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Different results across KM instances #928

Open
nurkmez2 opened this issue Dec 5, 2024 · 2 comments
Open

[Bug] Different results across KM instances #928

nurkmez2 opened this issue Dec 5, 2024 · 2 comments
Labels
bug Something isn't working triage

Comments

@nurkmez2
Copy link

nurkmez2 commented Dec 5, 2024

Context / Scenario

Scenario;

I have 3 different platforms;

  • A kernel memory service running in a local docker
  • A Kernel memory service tane azure app container
  • A dotnet serverless app running with Azure Search Service

All settings are the same for all 3 platform

SearchAsync finds different results.

What happened?

When SearchAsync is called with minRelevance: 0.6 for all 3 platforms above,

  • The service running in local docker finds no results.
  • But the service running in Azure app container and dotnet serverless app find 10 results.

**When I decrease minRelevance: 0.01,

  • All find 10 results.

What can cause this result?

Importance

a fix would make my life easier

Platform, Language, Versions

Windows, C#, last version of KM

Relevant log output

@nurkmez2 nurkmez2 added bug Something isn't working triage labels Dec 5, 2024
@dluc
Copy link
Collaborator

dluc commented Dec 6, 2024

The service has some defaults that you might not have in the serverless, so I would compare these configurations:

  • KernelMemory.DataIngestion.TextPartitioning
  • KernelMemory.Retrieval.SearchClient
  • KernelMemory.Services.AzureAISearch

Do you have 3 different results or 2 different results? Could you provide more details about the differences, and about the configurations used? Are you using hybrid search in any of the 3?

@dluc dluc changed the title [Bug] [Bug] Different results across KM instances Dec 6, 2024
@dluc dluc added the waiting for author Waiting for author to reply or address comments label Dec 6, 2024
@nurkmez2
Copy link
Author

nurkmez2 commented Dec 6, 2024

Thanks for the prompt reply
The result only differs if I send the same query to the Kernel Memory Service running Locally (Item -1). The rest have the same result.

1. configurations for kernel memory service running in a local docker
Image

Logs from this service
Image

  1. configurations for the kernel memory service running in the Azure app container, the same as the kernel memory service running in a local docker

Just in case some manual entry
Image

3. The dotnet serverless app running with Azure Search Service

Image

@dluc dluc removed the waiting for author Waiting for author to reply or address comments label Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

2 participants