Feature/make parsing chunking providers

[emrgnt-cmplxty]

🚀	This description was created by Ellipsis for commit 9bfef06

Summary:

Introduced chunking and parsing providers, updated configurations, and modified pipelines and tests to integrate new providers.

Key points:

• Added ChunkingProvider and ParsingProvider classes in r2r/base/providers/chunking.py and r2r/base/providers/parsing.py.
• Updated r2r.toml and pyproject.toml to include new chunking and parsing configurations.
• Modified r2r/base/__init__.py to import new providers.
• Created ChunkingPipe and ParsingPipe in r2r/pipes/ingestion.
• Updated IngestionPipeline in r2r/pipelines/ingestion_pipeline.py to use new pipes.
• Added R2RChunkingProvider and UnstructuredChunkingProvider in r2r/providers/chunking.
• Added R2RParsingProvider and UnstructuredParsingProvider in r2r/providers/parsing.
• Updated R2RConfig in r2r/main/assembly/config.py to handle new configurations.
• Updated R2RProviderFactory in r2r/main/assembly/factory.py to create new providers.
• Updated tests in tests/test_config.py to cover new configurations.

---

Generated with ❤️ by ellipsis.dev

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to 9bfef06 in 54 seconds

More details

• Looked at 1900 lines of code in 26 files
• Skipped 1 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. r2r.toml:10

• Draft comment:
  The chunking configuration in r2r.toml correctly specifies the unstructured provider and the by_title method, which aligns with the capabilities introduced in the PR. This is a crucial update ensuring that the new chunking functionality is configurable.
• Reason this comment was not posted:
  Confidence changes required: 0%
  The PR introduces a new chunking provider using the unstructured library, which is added to the dependencies in pyproject.toml. The r2r.toml configuration file is updated to include settings for this new chunking provider, specifying methods like by_title which aligns with the capabilities of the unstructured library. The codebase includes new classes for handling chunking operations, specifically R2RChunkingProvider and UnstructuredChunkingProvider, which are correctly implemented to handle the respective chunking logic based on the configuration. The PR also refactors the parsing configuration into its own section in r2r.toml, moving away from the previous ingestion section, which is a logical change to separate concerns of parsing and chunking from other types of data ingestion.

Workflow ID: wflow_mGiWs0CZ0xfjNbW6

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.