Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: add new clean up strategy "scoped_full" to indexing #28505

Merged
merged 8 commits into from
Dec 13, 2024
Prev Previous commit
Next Next commit
Update libs/core/langchain_core/indexing/api.py
  • Loading branch information
eyurtsev authored Dec 13, 2024
commit decf48e1e52a20093a9cdda110699fe1d1921f4f
2 changes: 1 addition & 1 deletion libs/core/langchain_core/indexing/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -504,7 +504,7 @@ async def aindex(
chunks, and we index them using a batch size of 5, we'll have 3 batches
all with the same source id. In general, to avoid doing too much
redundant work select as big a batch size as possible.
* scoped_full mode may be a good option if you have difficulty decidng
* scoped_full mode may be a good option if you have difficulty deciding
on the appropriate batch size and your loader is not able to
return the entire dataset. This solution keeps track of the source ids
in memory. It would probably be fine for most use cases in terms of
Expand Down
Loading