Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: update components to v1.0.18 #3903

Merged
merged 4 commits into from
Sep 24, 2024
Merged

Docs: update components to v1.0.18 #3903

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
daa7c66
embeddings
mendonk Sep 24, 2024
7465a42
helpers
mendonk Sep 24, 2024
4b3ac8e
vector-stores
mendonk Sep 24, 2024
2ee4465
hashes
mendonk Sep 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 101 additions & 25 deletions docs/docs/Components/components-embedding-models.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,22 +48,24 @@ This component is used to load embedding models from [Amazon Bedrock](https://aw
|------|------|-------------|
| embeddings | Embeddings | An instance for generating embeddings using Amazon Bedrock |

## Astra DB vectorize

## Astra vectorize
Connect this component to the **Embeddings** port of the [Astra DB vector store component](components-vector-stores#astra-db-serverless) to generate embeddings.

This component is used to generate server-side embeddings using [DataStax Astra](https://docs.datastax.com/en/astra-db-serverless/databases/embedding-generation.html).
This component requires that your Astra DB database has a collection that uses a vectorize embedding provider integration.
For more information and instructions, see [Embedding Generation](https://docs.datastax.com/en/astra-db-serverless/databases/embedding-generation.html).

### Parameters

#### Inputs

| Name | Type | Description |
|------|------|-------------|
| provider | String | The embedding provider to use |
| model_name | String | The embedding model to use |
| authentication | Dict | Authentication parameters. Use the Astra Portal to add the embedding provider integration to your Astra organization |
| provider_api_key | String | An alternative to the Astra Authentication that lets you use directly the API key of the provider |
| model_parameters | Dict | Additional model parameters |
| Name | Display Name | Info |
|------|--------------|------|
| provider | Embedding Provider | The embedding provider to use |
| model_name | Model Name | The embedding model to use |
| authentication | Authentication | The name of the API key in Astra that stores your [vectorize embedding provider credentials](https://docs.datastax.com/en/astra-db-serverless/databases/embedding-generation.html#embedding-provider-authentication). (Not required if using an [Astra-hosted embedding provider](https://docs.datastax.com/en/astra-db-serverless/databases/embedding-generation.html#supported-embedding-providers).) |
| provider_api_key | Provider API Key | As an alternative to `authentication`, directly provide your embedding provider credentials. |
| model_parameters | Model Parameters | Additional model parameters |

#### Outputs

Expand Down Expand Up @@ -112,29 +114,86 @@ This component is used to load embedding models from [Cohere](https://cohere.com
|------|------|-------------|
| embeddings | Embeddings | An instance for generating embeddings using Cohere |

## Hugging Face Inference API Embeddings
## Embedding similarity

This component generates embeddings using Hugging Face Inference API models.
This component computes selected forms of similarity between two embedding vectors.

### Parameters

#### Inputs

| Name | Type | Description |
|------|------|-------------|
| API Key | String | API key for accessing the Hugging Face Inference API |
| API URL | String | URL of the Hugging Face Inference API (default: `http://localhost:8080`) |
| Model Name | String | Name of the model to use for embeddings (default: `BAAI/bge-large-en-v1.5`) |
| Cache Folder | String | Folder path to cache Hugging Face models |
| Encode Kwargs | Dict | Additional arguments for the encoding process |
| Model Kwargs | Dict | Additional arguments for the model |
| Multi Process | Boolean | Whether to use multiple processes (default: `False`) |
| Name | Display Name | Info |
|------|--------------|------|
| embedding_vectors | Embedding Vectors | A list containing exactly two data objects with embedding vectors to compare. |
| similarity_metric | Similarity Metric | Select the similarity metric to use. Options: "Cosine Similarity", "Euclidean Distance", "Manhattan Distance". |

#### Outputs

| Name | Type | Description |
|------|------|-------------|
| embeddings | Embeddings | An instance for generating embeddings using Hugging Face Inference API |
| Name | Display Name | Info |
|------|--------------|------|
| similarity_data | Similarity Data | Data object containing the computed similarity score and additional information. |

## Google generative AI embeddings

This component connects to Google's generative AI embedding service using the GoogleGenerativeAIEmbeddings class from the `langchain-google-genai` package.

### Parameters

#### Inputs

| Name | Display Name | Info |
|------|--------------|------|
| api_key | API Key | Secret API key for accessing Google's generative AI service (required) |
| model_name | Model Name | Name of the embedding model to use (default: "models/text-embedding-004") |

#### Outputs

| Name | Display Name | Info |
|------|--------------|------|
| embeddings | Embeddings | Built GoogleGenerativeAIEmbeddings object |

## Hugging Face Embeddings

:::note
This component is deprecated as of Langflow version 1.0.18.
Instead, use the [Hugging Face API Embeddings component](#hugging-face-embeddings-inference-api).
:::

This component loads embedding models from HuggingFace.

Use this component to generate embeddings using locally downloaded Hugging Face models. Ensure you have sufficient computational resources to run the models.

### Parameters

#### Inputs

| Name | Display Name | Info |
|------|--------------|------|
| Cache Folder | Cache Folder | Folder path to cache HuggingFace models |
| Encode Kwargs | Encoding Arguments | Additional arguments for the encoding process |
| Model Kwargs | Model Arguments | Additional arguments for the model |
| Model Name | Model Name | Name of the HuggingFace model to use |
| Multi Process | Multi-Process | Whether to use multiple processes |

## Hugging Face embeddings Inference API

This component generates embeddings using Hugging Face Inference API models.

Use this component to create embeddings with Hugging Face's hosted models. Ensure you have a valid Hugging Face API key.

### Parameters

#### Inputs

| Name | Display Name | Info |
|------|--------------|------|
| API Key | API Key | API key for accessing the Hugging Face Inference API |
| API URL | API URL | URL of the Hugging Face Inference API |
| Model Name | Model Name | Name of the model to use for embeddings |
| Cache Folder | Cache Folder | Folder path to cache Hugging Face models |
| Encode Kwargs | Encoding Arguments | Additional arguments for the encoding process |
| Model Kwargs | Model Arguments | Additional arguments for the model |
| Multi Process | Multi-Process | Whether to use multiple processes |

## MistralAI

Expand Down Expand Up @@ -239,6 +298,25 @@ This component is used to load embedding models from [OpenAI](https://openai.com
|------|------|-------------|
| embeddings | Embeddings | An instance for generating embeddings using OpenAI |

## Text embedder

This component generates embeddings for a given message using a specified embedding model.

### Parameters

#### Inputs

| Name | Display Name | Info |
|------|--------------|------|
| embedding_model | Embedding Model | The embedding model to use for generating embeddings. |
| message | Message | The message for which to generate embeddings. |

#### Outputs

| Name | Display Name | Info |
|------|--------------|------|
| embeddings | Embedding Data | Data object containing the original text and its embedding vector. |

## VertexAI Embeddings

This component is a wrapper around [Google Vertex AI](https://cloud.google.com/vertex-ai) [Embeddings API](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings).
Expand Down Expand Up @@ -267,5 +345,3 @@ This component is a wrapper around [Google Vertex AI](https://cloud.google.com/v
|------|------|-------------|
| embeddings | Embeddings | An instance for generating embeddings using VertexAI |

[Previous Vector Stores](/components-vector-stores)

57 changes: 56 additions & 1 deletion docs/docs/Components/components-helpers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,26 @@ This component dynamically creates a record with a specified number of fields.

Use this component as a template to create your custom component.

For more, see [Custom Components](components-custom-components).

## Filter Data

This component converts LangChain documents into Data.
This component filters a Data object based on a list of keys.

### Parameters

#### Inputs

| Name | Display Name | Info |
|------|--------------|------|
| data | Data | Data object to filter. |
| filter_criteria | Filter Criteria | List of keys to filter by. |

#### Outputs

| Name | Display Name | Info |
|------|--------------|------|
| filtered_data | Filtered Data | A new Data object containing only the key-value pairs that match the filter criteria. |

## Hierarchical Task

Expand Down Expand Up @@ -103,8 +120,46 @@ This component generates a unique ID.
|------|--------------|------|
| value | Value | Unique ID generated. |

## Parse JSON

This component converts and extracts JSON fields using JQ queries.

### Parameters

#### Inputs

| Name | Display Name | Info |
|------|--------------|------|
| input_value | Input | Data object to filter. Can be a Message or Data object. |
| query | JQ Query | JQ Query to filter the data. The input is always a JSON list. |

#### Outputs

| Name | Display Name | Info |
|------|--------------|------|
| filtered_data | Filtered Data | Filtered data as a list of Data objects. |

## Merge Data

This component combines multiple data sources into a single unified Data object.

The component iterates through the input list of Data objects, merging them into a single Data object. If the input list is empty, it returns an empty Data object. If there's only one input Data object, it returns that object unchanged. The merging process uses the addition operator to combine Data objects.

### Parameters

#### Inputs

| Name | Display Name | Info |
|------|--------------|------|
| data | Data | A list of Data objects to be merged |

#### Outputs

| Name | Display Name | Info |
|------|--------------|------|
| merged_data | Merged Data | A single Data object containing the combined information from all input Data objects |


## Parse Data

The ParseData component converts Data objects into plain text using a specified template.
Expand Down
Loading
Loading