Skip to content

chore: CVE upgrades for wandb, gitpython and cleanups #1251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mckornfield wants to merge 5 commits into
base: main
Choose a base branch
from
Open

chore: CVE upgrades for wandb, gitpython and cleanups #1251

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
38c16ba
chore: CVE upgrades for wandb, gitpython and cleanups
mckornfield Aug 12, 2026
d208961
chore: harden docker artifact downloads
mckornfield Aug 12, 2026
2476aa3
chore: reuse OPA fallback downloader in docker builds
mckornfield Aug 12, 2026
1d9b990
fix: safe-synthesizer main fix and retry fix
mckornfield Aug 12, 2026
b67b648
chore: notice file update
mckornfield Aug 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions NOTICE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,13 +123,13 @@ Ray-bundled JAXB runtime dependencies
wandb-core MPL-2.0 Go dependencies
Scope: Unmodified Go module dependencies linked into the wandb-core
executable distributed with the wandb Python package in container images.
wandb package version: 0.28.1
wandb package version: 0.28.2
wandb source distribution:
https://files.pythonhosted.org/packages/92/fb/8d3f96a8b143060d6fa145462d0785981373e04694e4152555ccb5d23939/wandb-0.28.1.tar.gz
https://files.pythonhosted.org/packages/fd/41/c2eb0e25e0504287442f0829a5dc380669b683f78068c79220acc626383f/wandb-0.28.2.tar.gz
wandb upstream source:
https://github.com/wandb/wandb
wandb-core source at reported build commit:
https://github.com/wandb/wandb/tree/v0.28.1
https://github.com/wandb/wandb/tree/v0.28.2
License: MPL-2.0
License text: LICENSES/MPL-2.0.txt

Expand Down
2 changes: 2 additions & 0 deletions docker/Dockerfile.nmp-api
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@ COPY --chown=1000:1000 --from=nmp-studio-ui /artifacts /static/studio
COPY --chown=1000:1000 --from=builder /app/plugins/nemo-data-designer/tiktoken-cache /app/tiktoken-cache
ENV TIKTOKEN_CACHE_DIR=/app/tiktoken-cache
COPY --chown=1000:1000 --from=builder /tmp/fastembed_cache /tmp/fastembed_cache
COPY docker/scripts/cve-cleanup.sh /bin/
RUN sh /bin/cve-cleanup.sh
ARG NMP_PLATFORM_VERSION=dev
ARG NMP_CODE_REVISION=
ENV NMP_PLATFORM_VERSION=${NMP_PLATFORM_VERSION}
Expand Down
2 changes: 1 addition & 1 deletion docker/Dockerfile.nmp-cpu-tasks
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ ENV TIKTOKEN_CACHE_DIR=/app/tiktoken-cache
ENV PATH="/app/.venv/bin:$PATH"
USER 0
COPY docker/scripts/cve-cleanup.sh /bin/
RUN bash /bin/cve-cleanup.sh
RUN sh /bin/cve-cleanup.sh
ENTRYPOINT ["nemo-platform"]
# Override CMD with "run" in orchestration (e.g., Helm, docker-compose)
CMD ["--help"]
9 changes: 8 additions & 1 deletion docker/Dockerfile.nmp-customizer-tasks
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ RUN --mount=type=cache,target=/root/.cache/uv \
"soupsieve>=2.8.4,<3" \
"tornado>=6.5.7,<7" \
"urllib3>=2.7.0,<3" \
"wandb==0.28.1"
"GitPython>=3.1.58,<4" \
"wandb==0.28.2"

# CVE cleanup for unused/stale packages from the NGC base image. The task entry
# points import from /opt/venv; scanners still report the shadowed system copies.
Expand All @@ -103,11 +104,15 @@ RUN apt-get update && \
/usr/local/lib/python3.12/dist-packages/grpc \
/usr/local/lib/python3.12/dist-packages/grpcio \
/usr/local/lib/python3.12/dist-packages/grpcio-*.dist-info \
/usr/local/lib/python3.12/dist-packages/flash_attn \
/usr/local/lib/python3.12/dist-packages/flash_attn-*.dist-info \
/usr/local/lib/python3.12/dist-packages/wandb \
/usr/local/lib/python3.12/dist-packages/wandb-*.dist-info \
/usr/local/lib/python3.12/dist-packages/mlflow \
/usr/local/lib/python3.12/dist-packages/mlflow-*.dist-info \
/usr/local/lib/python3.12/dist-packages/mlflow_skinny-*.dist-info \
/usr/local/lib/python3.12/dist-packages/git \
/usr/local/lib/python3.12/dist-packages/gitpython-*.dist-info \
/usr/local/lib/python3.12/dist-packages/mistune \
/usr/local/lib/python3.12/dist-packages/mistune-*.dist-info \
/usr/local/lib/python3.12/dist-packages/PIL \
Expand All @@ -117,6 +122,8 @@ RUN apt-get update && \
/usr/local/lib/python3.12/dist-packages/soupsieve-*.dist-info \
/usr/local/lib/python3.12/dist-packages/jupyter_server \
/usr/local/lib/python3.12/dist-packages/jupyter_server-*.dist-info \
/usr/local/lib/python3.12/dist-packages/jupyterlab \
/usr/local/lib/python3.12/dist-packages/jupyterlab-*.dist-info \
/usr/local/cuda/NsightSystems-cli-* \
/usr/local/bin/nsys \
/usr/local/cuda/bin/nsys
Expand Down
3 changes: 3 additions & 0 deletions docker/Dockerfile.nmp-unsloth-training
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,7 @@ RUN --mount=type=cache,target=/root/.cache/uv \
"mistune>=3.3.3,<4" \
"pillow>=12.3.0,<13" \
"soupsieve>=2.8.4,<3" \
"GitPython>=3.1.58,<4" \
"mlflow-skinny>=3.11.1,<3.12.0"

# Stale NGC system site-packages (verified on nvcr.io/nvidia/pytorch:26.05-py3).
Expand Down Expand Up @@ -222,6 +223,8 @@ RUN rm -rf \
/usr/local/lib/python3.12/dist-packages/soupsieve-*.dist-info \
/usr/local/lib/python3.12/dist-packages/jupyter_server \
/usr/local/lib/python3.12/dist-packages/jupyter_server-*.dist-info \
/usr/local/lib/python3.12/dist-packages/jupyterlab \
/usr/local/lib/python3.12/dist-packages/jupyterlab-*.dist-info \
/usr/local/cuda/NsightSystems-cli-* \
/usr/local/bin/nsys \
/usr/local/cuda/bin/nsys
Expand Down
4 changes: 2 additions & 2 deletions docker/Dockerfile.safe-synthesizer-tasks
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ RUN printf '%s\n' \
"${SAFE_SYNTHESIZER_RUNTIME_PACKAGE}" \
> /tmp/safe-synthesizer-runtime.txt && \
printf '%s\n' \
wandb==0.28.1 \
wandb==0.28.2 \
'aiohttp>=3.14.3,<4' \
'cryptography>=50.0.0,<51' \
'pyarrow>=23.0.1,<24' \
Expand Down Expand Up @@ -195,7 +195,7 @@ COPY docker/scripts/cve-cleanup.sh /bin/
# Perl CVE cleanup removes dpkg-dev/build-essential; keep the non-Perl
# compiler toolchain available for runtime JIT/native extension builds.
RUN apt-mark manual gcc g++ make libc6-dev && \
bash /bin/cve-cleanup.sh && \
sh /bin/cve-cleanup.sh && \
printf 'int main(void) { return 0; }\n' | gcc -x c - -o /tmp/gcc-check && \
rm -f /tmp/gcc-check
RUN mkdir -p "${XDG_CACHE_HOME}" "${MPLCONFIGDIR}" "${OUTLINES_CACHE_DIR}" && \
Expand Down
8 changes: 6 additions & 2 deletions docker/automodel/Dockerfile.nmp-automodel-base
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ RUN --mount=type=cache,target=/root/.cache/uv \
uv pip install \
"aiohttp>=3.13.3,<4" \
"black>=26.3.1,<27" \
"GitPython>=3.1.57,<4" \
"GitPython>=3.1.58,<4" \
"jaraco-context>=6.1.0,<7" \
"jupyter-server>=2.20.0,<3" \
"jupyterlab>=4.6.1,<5" \
Expand All @@ -111,7 +111,7 @@ RUN --mount=type=cache,target=/root/.cache/uv \
"urllib3>=2.7.0,<3" \
"mlflow-skinny>=3.11.1,<3.12.0" \
"grpcio>=1.81.1,<2" \
"wandb==0.28.1"
"wandb==0.28.2"

# Published base image (same filesystem as builder).
FROM ${PYTORCH_BASE} AS nmp-automodel-base
Expand Down Expand Up @@ -153,6 +153,8 @@ RUN rm -rf \
/usr/local/lib/python3.12/dist-packages/mlflow \
/usr/local/lib/python3.12/dist-packages/mlflow-*.dist-info \
/usr/local/lib/python3.12/dist-packages/mlflow_skinny-*.dist-info \
/usr/local/lib/python3.12/dist-packages/git \
/usr/local/lib/python3.12/dist-packages/gitpython-*.dist-info \
/usr/local/lib/python3.12/dist-packages/PIL \
/usr/local/lib/python3.12/dist-packages/pillow.libs \
/usr/local/lib/python3.12/dist-packages/pillow-*.dist-info \
Expand All @@ -162,6 +164,8 @@ RUN rm -rf \
/usr/local/lib/python3.12/dist-packages/soupsieve-*.dist-info \
/usr/local/lib/python3.12/dist-packages/jupyter_server \
/usr/local/lib/python3.12/dist-packages/jupyter_server-*.dist-info \
/usr/local/lib/python3.12/dist-packages/jupyterlab \
/usr/local/lib/python3.12/dist-packages/jupyterlab-*.dist-info \
/usr/local/cuda/NsightSystems-cli-* \
/usr/local/bin/nsys \
/usr/local/cuda/bin/nsys
Expand Down
2 changes: 1 addition & 1 deletion docker/automodel/no_override_requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ safetensors; sys_platform == 'never'
numpy; sys_platform == 'never'

# Keep W&B's bundled wandb-core binary on a build with patched go-git/go-billy.
wandb==0.28.1
wandb==0.28.2
17 changes: 1 addition & 16 deletions docker/base/Dockerfile.policy-wasm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,10 @@
FROM alpine:3.19 AS root-policy-wasm-builder
RUN apk add --no-cache curl tar
ARG OPA_VERSION=v1.8.0
ARG TARGETARCH
RUN set -eux; \
case "${TARGETARCH}" in \
amd64|arm64) opa_arch="${TARGETARCH}" ;; \
"") opa_arch="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/aarch64/arm64/')" ;; \
*) echo "Unsupported architecture: ${TARGETARCH}" >&2; exit 1 ;; \
esac; \
opa_asset="opa_linux_${opa_arch}_static"; \
opa_url="https://github.com/open-policy-agent/opa/releases/download/${OPA_VERSION}/${opa_asset}"; \
curl -fsSL -o "/tmp/${opa_asset}" "${opa_url}"; \
curl -fsSL -o "/tmp/${opa_asset}.sha256" "${opa_url}.sha256"; \
(cd /tmp && sha256sum -c "${opa_asset}.sha256"); \
chmod +x "/tmp/${opa_asset}"; \
mv "/tmp/${opa_asset}" /usr/local/bin/opa; \
rm -f "/tmp/${opa_asset}.sha256"
WORKDIR /build
COPY script/build_policy_wasm.sh /build/script/
COPY services/core/auth/src/nmp/core/auth/app/policies /build/services/core/auth/src/nmp/core/auth/app/policies
RUN REPO_ROOT=/build OUTPUT_DIR=/artifacts sh /build/script/build_policy_wasm.sh
RUN OPA_VERSION="${OPA_VERSION}" REPO_ROOT=/build OUTPUT_DIR=/artifacts sh /build/script/build_policy_wasm.sh

FROM scratch AS root-policy-wasm-artifacts
COPY --from=root-policy-wasm-builder /artifacts/policy.wasm /artifacts/policy.wasm
55 changes: 49 additions & 6 deletions docker/rl/Dockerfile.nmp-rl-base
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,13 @@ ARG PYTHON_VERSION
# UV_PYTHON pins the interpreter for every uv call, at build and at runtime (node-built venvs too).
# Required, not cosmetic: RL ships a `.python-version` that otherwise wins, so bumping PYTHON_VERSION
# alone leaves every venv on RL's patch release and the requested one unused on disk.
# Raise uv's HTTP retry budget and cap concurrent downloads for large GitHub-hosted wheels like vLLM,
# which can hit transient HTTP/2 refused-stream failures under BuildKit.
ENV UV_PYTHON_INSTALL_DIR=/opt/uv/python \
UV_PYTHON=${PYTHON_VERSION} \
UV_HTTP_TIMEOUT=120 \
UV_HTTP_RETRIES=8 \
UV_CONCURRENT_DOWNLOADS=4 \
PATH="/usr/local/bin:/root/.local/bin:${PATH}"
RUN curl -LsSf "https://astral.sh/uv/${UV_VERSION}/install.sh" | sh && \
cp /root/.local/bin/uv /usr/local/bin/uv && \
Expand Down Expand Up @@ -182,18 +187,41 @@ RUN <<"EOF" bash -exu
export UV_LINK_MODE=symlink
uv venv --seed

uv sync ${UV_SYNC_MODE} --no-install-project
uv sync ${UV_SYNC_MODE} --extra vllm --no-install-project # GRPO generation (vllm 0.20 cu130 + flashinfer + deep_gemm/deep_ep)
uv sync ${UV_SYNC_MODE} --extra fsdp --no-install-project # DPO/GRPO policy training (flash-attn, mamba-ssm, causal-conv1d)
uv sync ${UV_SYNC_MODE} --extra modelopt --no-install-project # quantization / model-opt
uv sync ${UV_SYNC_MODE} --extra nemo_gym --no-install-project # NeMo-Gym (uv workspace member)
uv sync ${UV_SYNC_MODE} --all-groups --no-install-project # build/test/etc groups
uv_sync() {
local attempt
local sleep_seconds

for attempt in 1 2 3; do
if uv sync "$@"; then
return 0
fi

if [ "${attempt}" -eq 3 ]; then
return 1
fi

sleep_seconds=$((attempt * 20))
echo "uv sync failed; retrying in ${sleep_seconds}s (attempt $((attempt + 1))/3)" >&2
sleep "${sleep_seconds}"
done
}

uv_sync ${UV_SYNC_MODE} --no-install-project
uv_sync ${UV_SYNC_MODE} --extra vllm --no-install-project # GRPO generation (vllm 0.20 cu130 + flashinfer + deep_gemm/deep_ep)
uv_sync ${UV_SYNC_MODE} --extra fsdp --no-install-project # DPO/GRPO policy training (flash-attn, mamba-ssm, causal-conv1d)
uv_sync ${UV_SYNC_MODE} --extra modelopt --no-install-project # quantization / model-opt
uv_sync ${UV_SYNC_MODE} --extra nemo_gym --no-install-project # NeMo-Gym (uv workspace member)
uv_sync ${UV_SYNC_MODE} --all-groups --no-install-project # build/test/etc groups

# Gym sandbox CLIENT SDK (OpenSandbox provider). The OpenSandbox SERVER is deployed separately on
# the cluster; this is the client the Gym stack imports to call it. Source-independent, so it stays
# in this cached compile layer (the editable ROOT install moves below, after the full-source COPY).
uv pip install --python /opt/nemo_rl_venv/bin/python "opensandbox>=0.1.9" "tenacity>=9.1.4"

# W&B ships a scanner-visible Go binary. Keep it above the fixed go-git/grpc
# and Go stdlib floors even when NeMo-RL's lock has an older wheel.
uv pip install --python /opt/nemo_rl_venv/bin/python "wandb==0.28.2"

# CVE GHSA-mqqc-3gqh-h2x8: drop ray's bundled old aiohttp from the uv cache.
find "${UV_CACHE_DIR}" -type d -path "*ray/_private/runtime_env/agent/thirdparty_files/aiohttp*" -exec rm -rf {} + || true

Expand Down Expand Up @@ -325,6 +353,21 @@ if [ -d "${GYM_CACHE}" ]; then
fi
echo "in-tree Gym cache is ${GYM_CACHE_MB} MB (metadata only) - ok"
fi

# Some actor/env venvs may contain W&B depending on the selected NeMo-RL extras.
# Upgrade those in place, then drop any now-unreferenced older W&B cache archive
# so scanners do not report stale wandb-core binaries under /opt/uv_cache.
for py in /opt/ray_venvs/*/bin/python /opt/gym_venvs/*/bin/python; do
[ -x "${py}" ] || continue
if "${py}" -c 'import importlib.util, sys; sys.exit(0 if importlib.util.find_spec("wandb") else 1)'; then
uv pip install --python "${py}" "wandb==0.28.2"
fi
done
for d in "${UV_CACHE_DIR}"/archive-v0/*/; do
ls "${d}"wandb-*.dist-info >/dev/null 2>&1 || continue
find /opt/nemo_rl_venv /opt/ray_venvs /opt/gym_venvs -lname "*$(basename "${d}")*" -print -quit 2>/dev/null | grep -q . && continue
rm -rf "${d}"
done
EOF

# Container fingerprint (matches NeMo-RL's release stage). With NRL_CONTAINER=1 set, nemo_rl's
Expand Down
12 changes: 9 additions & 3 deletions docker/scripts/cve-cleanup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/sh

set -e

Expand All @@ -14,7 +14,7 @@ set -e
# binaries does not remove the scanner finding.
for venv in /app/.venv /opt/venv; do
if [ -d "${venv}" ]; then
if [ -x "${venv}/bin/python" ]; then
if [ -x "${venv}/bin/python" ] && command -v uv >/dev/null 2>&1; then
uv pip uninstall --python "${venv}/bin/python" \
opencv-contrib-python \
opencv-contrib-python-headless \
Expand All @@ -23,7 +23,9 @@ for venv in /app/.venv /opt/venv; do
fi

# Fallback for other packages that ship standalone ffmpeg binaries.
find "${venv}" -type f \( -name ffmpeg -o -name 'ffmpeg-*' \) -delete
if command -v find >/dev/null 2>&1; then
find "${venv}" -type f \( -name ffmpeg -o -name 'ffmpeg-*' \) -delete
fi
fi
done

Expand Down Expand Up @@ -87,6 +89,10 @@ rm -rf /usr/local/lib/python3.12/dist-packages/urllib3 \
rm -rf /usr/local/lib/python3.12/dist-packages/mlflow \
/usr/local/lib/python3.12/dist-packages/mlflow-*.dist-info \
/usr/local/lib/python3.12/dist-packages/mlflow_skinny-*.dist-info
rm -rf /usr/local/lib/python3.12/dist-packages/git \
/usr/local/lib/python3.12/dist-packages/gitpython-*.dist-info
rm -rf /usr/local/lib/python3.12/dist-packages/jupyterlab \
/usr/local/lib/python3.12/dist-packages/jupyterlab-*.dist-info
Comment on lines +92 to +95

@coderabbitai coderabbitai Bot Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Use the runtime Python path for the cleanup entries.

docker/Dockerfile.nmp-api and docker/Dockerfile.safe-synthesizer-tasks use Python 3.13, but Lines 90-93 only target /usr/local/lib/python3.12/dist-packages. These image builds can therefore skip the new GitPython and JupyterLab cleanup.

Discover the runtime site-packages directory or pass it from each image. Add a build assertion that the targeted files are absent.

Proposed fix 
-rm -rf /usr/local/lib/python3.12/dist-packages/git \
-       /usr/local/lib/python3.12/dist-packages/gitpython-*.dist-info
-rm -rf /usr/local/lib/python3.12/dist-packages/jupyterlab \
-       /usr/local/lib/python3.12/dist-packages/jupyterlab-*.dist-info
+for site_packages in \
+    /usr/local/lib/python*/dist-packages \
+    /usr/local/lib/python*/site-packages; do
+    [ -d "$site_packages" ] || continue
+    rm -rf \
+        "$site_packages/git" \
+        "$site_packages"/gitpython-*.dist-info \
+        "$site_packages/jupyterlab" \
+        "$site_packages"/jupyterlab-*.dist-info
+done
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docker/scripts/cve-cleanup.sh` around lines 90 - 93, Update the cleanup
entries in the script around the GitPython and JupyterLab removals to target the
runtime Python site-packages directory rather than hardcoded Python 3.12 paths.
Ensure Dockerfile.nmp-api and Dockerfile.safe-synthesizer-tasks provide or
resolve the correct Python 3.13 path, and add build assertions verifying the
targeted packages and dist-info files are absent afterward.

rm -rf /usr/local/lib/python3.12/dist-packages/PIL \
/usr/local/lib/python3.12/dist-packages/pillow.libs \
/usr/local/lib/python3.12/dist-packages/pillow-*.dist-info
2 changes: 1 addition & 1 deletion docker/unsloth/no_override_requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ accelerate; sys_platform == 'never'
safetensors; sys_platform == 'never'

# Keep W&B's bundled wandb-core binary on a build with patched go-git/go-billy.
wandb==0.28.1
wandb==0.28.2
8 changes: 6 additions & 2 deletions ...nemo-safe-synthesizer/src/nemo_safe_synthesizer_plugin/tasks/safe_synthesizer/__main__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -354,9 +354,13 @@ def run_config(
else:
ss.process_data()
total_time = time.monotonic() - (ss._total_start or time.monotonic())
pii_replaced_df = getattr(ss, "_train_df", None)
pii_replaced_df = getattr(ss, "_training_df", None)
if pii_replaced_df is None:
raise RuntimeError("process_data() completed but _train_df is None")
# Older upstream builds used _train_df; keep a fallback so
# plugin-local runs do not depend on exactly one NSS internals name.
pii_replaced_df = getattr(ss, "_train_df", None)
if pii_replaced_df is None:
raise RuntimeError("process_data() completed but processed training data is unavailable")
from nemo_safe_synthesizer.results import make_nss_results

ss.results = make_nss_results(generate_results=pii_replaced_df, total_time=total_time)
Expand Down
48 changes: 48 additions & 0 deletions plugins/nemo-safe-synthesizer/tests/unit/test_local_run.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,54 @@ def test_validate_flat_tabular_data_allows_flat_columns(monkeypatch):
task_main._validate_flat_tabular_data(data)


def test_run_config_pii_only_uses_processed_training_df(tmp_path, monkeypatch):
task_main = import_task_main_without_heavy_runtime(monkeypatch)
processed = pd.DataFrame({"name": ["REDACTED"], "age": [1]})

class FakeSafeSynthesizer:
def __init__(self, config, save_path):
self._total_start = None
self._training_df = None
self._workdir = None

def with_data_source(self, data_source):
return self

def process_data(self):
self._training_df = processed

monkeypatch.setattr(task_main, "SafeSynthesizer", FakeSafeSynthesizer)
results_module = ModuleType("nemo_safe_synthesizer.results")
setattr(
results_module,
"make_nss_results",
lambda generate_results, total_time=None: SimpleNamespace(
synthetic_data=generate_results,
summary=SimpleNamespace(model_dump=lambda: {"row_count": len(generate_results)}),
evaluation_report_html=None,
),
)
monkeypatch.setitem(sys.modules, "nemo_safe_synthesizer.results", results_module)
job_config = task_main.SafeSynthesizerJobConfig.model_validate(
{
"data_source": "default/data#input.csv",
"config": {
"enable_synthesis": False,
"enable_replace_pii": False,
},
}
)

result, adapter_path = task_main.run_config(
job_config,
pd.DataFrame({"name": ["Alice"], "age": [1]}),
tmp_path,
)

assert adapter_path is None
pd.testing.assert_frame_equal(result.synthetic_data, processed)


def test_run_from_env_reports_missing_config_path(monkeypatch):
task_main = import_task_main_without_heavy_runtime(monkeypatch)
monkeypatch.setattr(task_main, "initialize_observability", lambda: None)
Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ environments = [
]

constraint-dependencies = [
"GitPython>=3.1.57",
"GitPython>=3.1.58",
"Mako>=1.3.12",
"Pygments>=2.20.0",
"aiohttp>=3.14.1", # High/Medium/Low OOS – auditor-tasks + customizer
Expand Down Expand Up @@ -340,7 +340,7 @@ override-dependencies = [
"ujson>=5.12.0", # High – nmp-api/cpu-tasks/gpu-tasks
"xgrammar>=0.1.32", # High – customizer + nmp-gpu-tasks
"fastmcp>=3.2.0", # GHSA-vv7q-7jx5-f767 (Critical) + GHSA-rww4-4w9c-7733 (High); overrides vendored sdk/python/nemo-platform <3 constraint
"wandb>=0.28.1", # wandb-core Go CVEs (nspect); unsloth + training images
"wandb>=0.28.2", # wandb-core Go CVEs (nspect); unsloth + training images
"cryptography>=50.0.0,<51", # GHSA-g6cj-pr64-35w5; overrides nvidia-nat-core's <47 cap after changelog review
"click>=8.2.0", # Below CVEs are based on constraints that garak has
"litellm>=1.83.10", # CVE-2026-42208 (Critical SQL injection), CVE-2026-40217, and 5 other High CVEs
Expand Down
Loading
Loading