Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core/indexer] Reverts PR #25754 and add unit tests #28702

Merged

Conversation

KeiichiHirobe
Copy link
Contributor

@KeiichiHirobe KeiichiHirobe commented Dec 13, 2024

I reported the bug 2 weeks ago here: #28447

I believe this is a critical bug for the indexer, so I submitted a PR to revert the change and added unit tests to prevent similar bugs from being introduced in the future.

@eyurtsev Could you check this?

Copy link

vercel bot commented Dec 13, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchain ⬜️ Ignored (Inspect) Visit Preview Dec 13, 2024 8:21am

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. 🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature labels Dec 13, 2024
@@ -630,26 +630,18 @@ def test_incremental_delete(
}


def test_incremental_indexing_with_batch_size(
def test_incremental_delete_with_same_source(
Copy link
Contributor Author

@KeiichiHirobe KeiichiHirobe Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, the diff in test_indexing.py is very confusing, but test_incremental_delete_with_same_source is the only test case I newly created. I recommend reviewing the diff for each commit.

"num_deleted": 0,
"num_skipped": 4,
"num_deleted": 1,
"num_skipped": 1,
Copy link
Contributor Author

@KeiichiHirobe KeiichiHirobe Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed that the original logic returned num_deleted = 0, which is an obviously unexpected result.

@eyurtsev
Copy link
Collaborator

Related to PR: #25754

@eyurtsev eyurtsev merged commit da28cf1 into langchain-ai:master Dec 13, 2024
81 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature size:L This PR changes 100-499 lines, ignoring generated files.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants