Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions catalog/plugins.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
},
{
"name": "data-designer-retrieval-sdg",
"description": "Retriever SDG toolkit: registers the embedding-dedup column generator and document-chunker seed reader, plus a multi-step QA generation pipeline, CLI, and Automodel-compatible data conversion",
"description": "Retriever SDG toolkit with Data Designer extensions, a multi-step QA generation pipeline, CLI, and Automodel-compatible data conversion",
"install": {
"requirement": "data-designer-retrieval-sdg",
"index_url": "https://nvidia-nemo.github.io/DataDesignerPlugins/simple/"
Expand All @@ -45,8 +45,8 @@
"specifier": ">=3.10"
},
"data_designer": {
"requirement": "data-designer>=0.9.0",
"specifier": ">=0.9.0",
"requirement": "data-designer>=0.9.1",
"specifier": ">=0.9.1",
"marker": null
}
},
Expand Down
4 changes: 2 additions & 2 deletions docs/plugins/data-designer-retrieval-sdg/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@

# data-designer-retrieval-sdg

Retriever SDG toolkit: registers the embedding-dedup column generator and document-chunker seed reader, plus a multi-step QA generation pipeline, CLI, and Automodel-compatible data conversion
Retriever SDG toolkit with Data Designer extensions, a multi-step QA generation pipeline, CLI, and Automodel-compatible data conversion

This page is generated from package metadata because the plugin has not added
custom documentation under `plugins/data-designer-retrieval-sdg/docs/` yet.

## Metadata

- Version: `0.1.0`
- Version: `0.2.0`
- Entry points: `document-chunker`, `embedding-dedup`

## Installation
Expand Down
4 changes: 2 additions & 2 deletions docs/plugins/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ Browse available Data Designer plugins by what they add to your data generation
<a class="plugin-doc-card" href="data-designer-retrieval-sdg/" aria-label="Open data-designer-retrieval-sdg documentation">
<span class="plugin-doc-card__header">
<span class="plugin-doc-card__title">data-designer-retrieval-sdg</span>
<span class="plugin-doc-card__version">v0.1.0</span>
<span class="plugin-doc-card__version">v0.2.0</span>
</span>
<span class="plugin-doc-card__description">Retriever SDG toolkit: registers the embedding-dedup column generator and document-chunker seed reader, plus a multi-step QA generation pipeline, CLI, and Automodel-compatible data conversion</span>
<span class="plugin-doc-card__description">Retriever SDG toolkit with Data Designer extensions, a multi-step QA generation pipeline, CLI, and Automodel-compatible data conversion</span>
<span class="plugin-doc-card__section">
<span class="plugin-doc-card__label">Entry points</span>
<span class="plugin-doc-card__chips"><span class="plugin-doc-chip">document-chunker</span><span class="plugin-doc-chip">embedding-dedup</span></span>
Expand Down
6 changes: 3 additions & 3 deletions plugins/data-designer-retrieval-sdg/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,9 @@ compatibility, partial-result cleanup, and the behavior of every resume mode. Th
plugin does not maintain a second resume state or inspect corpus bytes.

`--buffer-size` controls DataDesigner's checkpoint/write granularity and remains
part of the resolved config. In DataDesigner 0.6.1, `create()` still profiles the
completed dataset before returning, so `--buffer-size` is not a hard cap on final
peak memory for very large runs.
part of the resolved config. DataDesigner still profiles the completed dataset
before returning, so `--buffer-size` is not a hard cap on final peak memory for
very large runs.

## Installation

Expand Down
10 changes: 7 additions & 3 deletions plugins/data-designer-retrieval-sdg/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,19 @@

[project]
name = "data-designer-retrieval-sdg"
version = "0.1.0"
description = "Retriever SDG toolkit: registers the embedding-dedup column generator and document-chunker seed reader, plus a multi-step QA generation pipeline, CLI, and Automodel-compatible data conversion"
version = "0.2.0"
description = "Retriever SDG toolkit with Data Designer extensions, a multi-step QA generation pipeline, CLI, and Automodel-compatible data conversion"
requires-python = ">=3.10"
dependencies = [
"data-designer>=0.9.0",
"data-designer>=0.9.1",
# Data Designer permits older transitive releases that are excluded by the
# workspace security constraints. Publish the same floors for consumers.
"mcp>=1.29.0,<2",
"nltk>=3.10.2,<3.11",
"pydantic-settings>=2.14,<3",
"pyyaml>=6.0",
"pyarrow>=24.0.0,<25",
"starlette>=1.4.1,<2",
]
license = "Apache-2.0"
readme = "README.md"
Expand Down
26 changes: 15 additions & 11 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading