Implement fetch_by_metadata for Index and IndexAsyncio

[jhamon]

Implement fetch_by_metadata for Index and IndexAsyncio

This PR adds the fetch_by_metadata method to both synchronous and asynchronous Pinecone index clients, allowing users to retrieve vectors based on metadata filters rather than requiring explicit vector IDs.

Overview

The fetch_by_metadata operation enables querying vectors by their metadata attributes, similar to how query works but without requiring a query vector. This is particularly useful for:

• Retrieving all vectors matching specific metadata criteria
• Building data pipelines that filter by metadata
• Implementing metadata-based data retrieval workflows

Usage Examples

Basic Usage (Synchronous)

from pinecone import Pinecone

pc = Pinecone(api_key='your-api-key')
index = pc.Index(host='your-index-host')

# Fetch vectors with simple metadata filter
result = index.fetch_by_metadata(
    filter={"genre": "action"},
    namespace="movies"
)

# Iterate over results
for vec_id, vector in result.vectors.items():
    print(f"ID: {vector.id}, Metadata: {vector.metadata}")

Complex Filtering

# Using multiple filter conditions
result = index.fetch_by_metadata(
    filter={
        "genre": {"$in": ["comedy", "drama"]},
        "year": {"$gte": 2020},
        "rating": {"$gt": 7.5}
    },
    namespace="movies",
    limit=100
)

Pagination

# First page
result = index.fetch_by_metadata(
    filter={"status": "active"},
    namespace="products",
    limit=50
)

# Continue to next page if available
if result.pagination and result.pagination.next:
    next_page = index.fetch_by_metadata(
        filter={"status": "active"},
        namespace="products",
        limit=50,
        pagination_token=result.pagination.next
    )

Asynchronous Usage

import asyncio
from pinecone import Pinecone

async def main():
    pc = Pinecone(api_key='your-api-key')
    async with pc.IndexAsyncio(host='your-index-host') as index:
        result = await index.fetch_by_metadata(
            filter={"category": "electronics", "in_stock": True},
            namespace="inventory",
            limit=100
        )
        
        for vec_id, vector in result.vectors.items():
            print(f"Product {vector.id}: {vector.metadata}")

asyncio.run(main())

gRPC Usage

from pinecone.grpc import PineconeGRPC

pc = PineconeGRPC(api_key='your-api-key')
index = pc.Index(host='your-index-host')

# Synchronous gRPC call
result = index.fetch_by_metadata(
    filter={"tag": "featured"},
    namespace="articles"
)

# Asynchronous gRPC call (returns future)
future = index.fetch_by_metadata(
    filter={"tag": "featured"},
    namespace="articles",
    async_req=True
)

# Wait for result
result = future.result()

Filter Operators

The fetch_by_metadata method supports all standard Pinecone metadata filter operators:

# Equality
filter={"status": "active"}

# Comparison operators
filter={"price": {"$gt": 100}}
filter={"age": {"$gte": 18}}
filter={"score": {"$lt": 0.5}}
filter={"count": {"$lte": 10}}

# Array operators
filter={"tags": {"$in": ["red", "blue", "green"]}}
filter={"categories": {"$nin": ["deprecated"]}}

# Existence check
filter={"description": {"$exists": True}}

# Logical operators
filter={
    "$and": [
        {"status": "active"},
        {"price": {"$lt": 50}}
    ]
}

filter={
    "$or": [
        {"category": "electronics"},
        {"category": "computers"}
    ]
}

Response Structure

The method returns a FetchByMetadataResponse object containing:

class FetchByMetadataResponse:
    namespace: str                    # The namespace queried
    vectors: Dict[str, Vector]        # Dictionary of vector ID to Vector objects
    usage: Usage                      # API usage information
    pagination: Optional[Pagination]  # Pagination token for next page (if available)

Technical Changes

Core Implementation

• Added fetch_by_metadata method to Index (sync) and _IndexAsyncio (async) classes
• Added fetch_by_metadata method to GRPCIndex with support for async_req
• Created FetchByMetadataResponse dataclass with pagination support
• Added request factory method IndexRequestFactory.fetch_by_metadata_request
• Added gRPC response parser parse_fetch_by_metadata_response

Protobuf Migration

• Migrated from db_data_2025_04 protobuf stubs to db_data_2025_10 stubs
• Updated all gRPC-related imports and references
• Removed deprecated 2025-04 stub files

Testing

• Added comprehensive integration tests for sync (test_fetch_by_metadata.py)
• Added comprehensive integration tests for async (test_fetch_by_metadata.py)
• Added gRPC futures tests (test_fetch_by_metadata_future.py)
• Added unit tests for request factory (test_request_factory.py)
• Added unit tests for Index class (test_index.py)
• Updated all unit test files to use 2025-10 protobuf stubs

Documentation

• Added usage examples to docs/db_data/index-usage-byov.md
• Updated interface docstrings with examples

Breaking Changes

None. This is a new feature addition.

Migration Notes

No migration required. This is a new feature that doesn't affect existing functionality.