diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6b6e8194e..f907ae5fc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -71,4 +71,4 @@ By making a contribution to this project, I certify that: this project or the open source license(s) involved. ``` -Note: This section `Sign your work` is derived from [https://github.com/NVIDIA/spark-rapids](https://github.com/NVIDIA/spark-rapids) +Note: This section `Sign your work` is derived from [https://github.com/NVIDIA/cudf-spark](https://github.com/NVIDIA/cudf-spark) diff --git a/README.md b/README.md index 865e9ef48..1b1dc3cac 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,15 @@ # spark-rapids-examples -This is the [RAPIDS Accelerator for Apache Spark](https://nvidia.github.io/spark-rapids/) examples repo. -RAPIDS Accelerator for Apache Spark accelerates Spark applications with no code changes. -You can download the latest version of RAPIDS Accelerator [here](https://nvidia.github.io/spark-rapids/docs/download.html). +This is the [NVIDIA cuDF plugin for Apache Spark](https://nvidia.github.io/cudf-spark/) examples repo. +The cuDF plugin accelerates Spark applications with no code changes. +You can download the latest version of the cuDF plugin [here](https://nvidia.github.io/cudf-spark/docs/download.html). This repo contains examples and applications that showcases the performance and benefits of using -RAPIDS Accelerator in data processing and machine learning pipelines. +the cuDF plugin in data processing and machine learning pipelines. There are broadly five categories of examples in this repo: 1. [SQL/Dataframe](./examples/SQL+DF-Examples) 2. [Spark XGBoost](./examples/XGBoost-Examples) 3. [Machine Learning/Deep Learning](./examples/ML+DL-Examples) -4. [RAPIDS UDF](./examples/UDF-Examples) +4. [cuDF plugin UDF](./examples/UDF-Examples) 5. [Databricks Tools demo notebooks](./tools/databricks) For more information on each of the examples please look into respective categories. @@ -29,7 +29,7 @@ Here is the list of notebooks in this repo: | 9 | SQL/DF | [TPC-DS](https://www.tpc.org/tpcds/) Scale Factor 10 | Comparison of Spark SQL CPU vs GPU. Easy to run locally and on Google Colab Here is the list of Apache Spark applications (Scala and PySpark) that -can be built for running on GPU with RAPIDS Accelerator in this repo: +can be built for running on GPU with the cuDF plugin in this repo: | | Category | Notebook Name | Description | ------------- | ------------- | ------------- | ------------- diff --git a/docs/get-started/xgboost-examples/csp/databricks/databricks.md b/docs/get-started/xgboost-examples/csp/databricks/databricks.md index cfbb838c5..64aa12eda 100644 --- a/docs/get-started/xgboost-examples/csp/databricks/databricks.md +++ b/docs/get-started/xgboost-examples/csp/databricks/databricks.md @@ -15,13 +15,13 @@ The number of GPUs per node dictates the number of Spark executors that can run Start A Databricks Cluster -------------------------- Before creating the cluster, we will need to create an [initialization script](https://docs.databricks.com/clusters/init-scripts.html) for the -cluster to install the RAPIDS jars. Databricks recommends storing all cluster-scoped init scripts using workspace files. +cluster to install the NVIDIA cuDF plugin for Apache Spark jar. Databricks recommends storing all cluster-scoped init scripts using workspace files. Each user has a Home directory configured under the /Users directory in the workspace. Navigate to your home directory in the UI and select **Create** > **File** from the menu, create an `init.sh` scripts with contents: ```bash #!/bin/bash - sudo wget -O /databricks/jars/rapids-4-spark_2.12-26.06.0.jar https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar + sudo wget -O /databricks/jars/rapids-4-spark_2.12-26.08.0.jar https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar ``` 1. Select the Databricks Runtime Version from one of the supported runtimes specified in the Prerequisites section. @@ -34,8 +34,7 @@ create an `init.sh` scripts with contents: ![Init Script](../../../../img/databricks/initscript.png) 6. Now select the “Spark” tab, and paste the following config options into the Spark Config section. Change the config values based on the workers you choose. See Apache Spark - [configuration](https://spark.apache.org/docs/latest/configuration.html) and RAPIDS Accelerator - for Apache Spark [descriptions](https://nvidia.github.io/spark-rapids/docs/configs.html) for each config. + [configuration](https://spark.apache.org/docs/latest/configuration.html) and the cuDF plugin [configuration descriptions](https://nvidia.github.io/cudf-spark/docs/configs.html) for each config. The [`spark.task.resource.gpu.amount`](https://spark.apache.org/docs/latest/configuration.html#scheduling) @@ -60,15 +59,15 @@ create an `init.sh` scripts with contents: of python for Databricks. On Databricks, the python runtime requires different parameters than the Spark one, so a dedicated python demon module `rapids.daemon_databricks` is created and should be specified here. Set the config - [`spark.rapids.sql.python.gpu.enabled`](https://nvidia.github.io/spark-rapids/docs/configs.html#sql.python.gpu.enabled) to `true` to + [`spark.rapids.sql.python.gpu.enabled`](https://nvidia.github.io/cudf-spark/docs/configs.html#sql.python.gpu.enabled) to `true` to enable GPU support for python. Add the path of the plugin jar (supposing it is placed under `/databricks/jars/`) to the `spark.executorEnv.PYTHONPATH` option. For more details please go to - [GPU Scheduling For Pandas UDF](https://nvidia.github.io/spark-rapids/docs/additional-functionality/rapids-udfs.html#gpu-support-for-pandas-udf) + [GPU Scheduling For Pandas UDF](https://nvidia.github.io/cudf-spark/docs/additional-functionality/rapids-udfs.html#gpu-support-for-pandas-udf) ```bash spark.rapids.sql.python.gpu.enabled true spark.python.daemon.module rapids.daemon_databricks - spark.executorEnv.PYTHONPATH /databricks/jars/rapids-4-spark_2.12-26.06.0.jar:/databricks/spark/python + spark.executorEnv.PYTHONPATH /databricks/jars/rapids-4-spark_2.12-26.08.0.jar:/databricks/spark/python ``` Note that since python memory pool require installing the cudf library, so you need to install cudf library in each worker nodes `pip install cudf-cu11 --extra-index-url=https://pypi.nvidia.com` or disable python memory pool @@ -166,6 +165,6 @@ Limitations 4. Databricks makes changes to the runtime without notification. Databricks makes changes to existing runtimes, applying patches, without notification. - [Issue-3098](https://github.com/NVIDIA/spark-rapids/issues/3098) is one example of this. We run + [Issue-3098](https://github.com/NVIDIA/cudf-spark/issues/3098) is one example of this. We run regular integration tests on the Databricks environment to catch these issues and fix them once detected. \ No newline at end of file diff --git a/docs/get-started/xgboost-examples/csp/databricks/init.sh b/docs/get-started/xgboost-examples/csp/databricks/init.sh index 2ce9c5eac..d8bcb447e 100644 --- a/docs/get-started/xgboost-examples/csp/databricks/init.sh +++ b/docs/get-started/xgboost-examples/csp/databricks/init.sh @@ -17,7 +17,7 @@ sudo rm -f /databricks/jars/spark--maven-trees--ml--10.x--xgboost-gpu--ml.dmlc--xgboost4j-gpu_2.12--ml.dmlc__xgboost4j-gpu_2.12__1.5.2.jar sudo rm -f /databricks/jars/spark--maven-trees--ml--10.x--xgboost-gpu--ml.dmlc--xgboost4j-spark-gpu_2.12--ml.dmlc__xgboost4j-spark-gpu_2.12__1.5.2.jar -sudo wget -O /databricks/jars/rapids-4-spark_2.12-26.06.0.jar https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar +sudo wget -O /databricks/jars/rapids-4-spark_2.12-26.08.0.jar https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar sudo wget -O /databricks/jars/xgboost4j-gpu_2.12-1.7.1.jar https://repo1.maven.org/maven2/ml/dmlc/xgboost4j-gpu_2.12/1.7.1/xgboost4j-gpu_2.12-1.7.1.jar sudo wget -O /databricks/jars/xgboost4j-spark-gpu_2.12-1.7.1.jar https://repo1.maven.org/maven2/ml/dmlc/xgboost4j-spark-gpu_2.12/1.7.1/xgboost4j-spark-gpu_2.12-1.7.1.jar ls -ltr diff --git a/docs/get-started/xgboost-examples/csp/dataproc/gcp.md b/docs/get-started/xgboost-examples/csp/dataproc/gcp.md index c01986cdc..8e25f13c6 100644 --- a/docs/get-started/xgboost-examples/csp/dataproc/gcp.md +++ b/docs/get-started/xgboost-examples/csp/dataproc/gcp.md @@ -1,4 +1,4 @@ -# Getting started pyspark+xgboost with RAPIDS Accelerator on GCP Dataproc +# Getting started with PySpark, XGBoost, and the NVIDIA cuDF plugin for Apache Spark on GCP Dataproc [Google Cloud Dataproc](https://cloud.google.com/dataproc) is Google Cloud's fully managed Apache Spark and Hadoop service. Please make sure to install gcloud CLI by following this [guide](https://cloud.google.com/sdk/docs/install) before getting started. diff --git a/docs/get-started/xgboost-examples/prepare-package-data/preparation-python.md b/docs/get-started/xgboost-examples/prepare-package-data/preparation-python.md index 0e9b55e88..c6dd816ad 100644 --- a/docs/get-started/xgboost-examples/prepare-package-data/preparation-python.md +++ b/docs/get-started/xgboost-examples/prepare-package-data/preparation-python.md @@ -4,8 +4,8 @@ For simplicity export the location to these jars. All examples assume the packag ### Download the jars -Download the RAPIDS Accelerator for Apache Spark plugin jar - * [RAPIDS Spark Package](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar) +Download the NVIDIA cuDF plugin for Apache Spark jar + * [cuDF plugin jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar) ### Build XGBoost Python Examples diff --git a/docs/get-started/xgboost-examples/prepare-package-data/preparation-scala.md b/docs/get-started/xgboost-examples/prepare-package-data/preparation-scala.md index d23d6115a..1c1d06525 100644 --- a/docs/get-started/xgboost-examples/prepare-package-data/preparation-scala.md +++ b/docs/get-started/xgboost-examples/prepare-package-data/preparation-scala.md @@ -4,8 +4,8 @@ For simplicity export the location to these jars. All examples assume the packag ### Download the jars -1. Download the RAPIDS Accelerator for Apache Spark plugin jar - * [RAPIDS Spark Package](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar) +1. Download the NVIDIA cuDF plugin for Apache Spark jar + * [cuDF plugin jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar) ### Build XGBoost Scala Examples diff --git a/examples/MIG-Support/device-plugins/gpu-mig/README.md b/examples/MIG-Support/device-plugins/gpu-mig/README.md index 942a626fd..611ceb564 100644 --- a/examples/MIG-Support/device-plugins/gpu-mig/README.md +++ b/examples/MIG-Support/device-plugins/gpu-mig/README.md @@ -103,5 +103,5 @@ Note the getMIGGpus discovery script would is in the `scripts` directory in this `nvidia/miggpu`. ## Testing -Run a Spark application using the [Rapids Accelerator for Apache Spark](https://nvidia.github.io/spark-rapids/) and request GPUs +Run a Spark application using the [NVIDIA cuDF plugin for Apache Spark](https://nvidia.github.io/cudf-spark/) and request GPUs from YARN and verify they use the MIG enabled GPUs. diff --git a/examples/MIG-Support/resource-types/gpu-mig/README.md b/examples/MIG-Support/resource-types/gpu-mig/README.md index 37142dff8..493ae8801 100644 --- a/examples/MIG-Support/resource-types/gpu-mig/README.md +++ b/examples/MIG-Support/resource-types/gpu-mig/README.md @@ -98,5 +98,5 @@ Environment variable for Spark application: ``` ## Testing -Run a Spark application using the [Rapids Accelerator for Apache Spark](https://nvidia.github.io/spark-rapids/) and request GPUs +Run a Spark application using the [NVIDIA cuDF plugin for Apache Spark](https://nvidia.github.io/cudf-spark/) and request GPUs from YARN and verify they use the MIG enabled GPUs. diff --git a/examples/ML+DL-Examples/Optuna-Spark/README.md b/examples/ML+DL-Examples/Optuna-Spark/README.md index 92bd7ee34..fd183a779 100644 --- a/examples/ML+DL-Examples/Optuna-Spark/README.md +++ b/examples/ML+DL-Examples/Optuna-Spark/README.md @@ -74,7 +74,7 @@ We provide **2 notebooks**, with differences in the backend/implementation. See - `optuna-dataframe.ipynb`: - Uses Spark dataframes to distribute tasks on the cluster. - Implements *Spark-I/O*, where Spark reads the dataset from a specified filepath, then duplicates and repartitions it so that each worker task is mapped onto a copy of the dataset. - - Dataframe operations are accelerated on GPU with the [Spark-RAPIDS Accelerator](https://nvidia.github.io/spark-rapids/). + - Dataframe operations are accelerated on GPU with the [NVIDIA cuDF plugin for Apache Spark](https://nvidia.github.io/cudf-spark/). ## Running Optuna on Spark Standalone @@ -147,8 +147,8 @@ We use [RAPIDS](https://docs.rapids.ai/install/#get-rapids) for GPU-accelerated ``` shell sudo apt install libmysqlclient-dev -conda create -n rapids-26.06 -c rapidsai -c conda-forge -c nvidia \ - cudf=26.06 cuml=26.06 python=3.10 'cuda-version>=12.0,<=12.5' +conda create -n rapids-26.08 -c rapidsai -c conda-forge -c nvidia \ + cudf=26.08 cuml=26.08 python=3.10 'cuda-version>=12.0,<=12.5' conda activate optuna-spark pip install mysqlclient pip install optuna joblib joblibspark ipywidgets @@ -201,7 +201,7 @@ Or, create a cluster via the web UI: - Under `Advanced Options > Spark > Environment variables`, set `LIBCUDF_CUFILE_POLICY=OFF`. - Make sure to use a GPU cluster and include task GPU resources. -The init script will install the required libraries on all nodes, including RAPIDS and the Spark-RAPIDS plugin for GPU-accelerated ETL. On the driver, it will setup the MySQL server backend. +The init script will install the required libraries on all nodes, including RAPIDS and the cuDF plugin for GPU-accelerated ETL. On the driver, it will setup the MySQL server backend. ### 3. Run Notebook diff --git a/examples/ML+DL-Examples/Optuna-Spark/optuna-examples/databricks/init_optuna.sh b/examples/ML+DL-Examples/Optuna-Spark/optuna-examples/databricks/init_optuna.sh index 69ab58982..dcfc5bbcf 100644 --- a/examples/ML+DL-Examples/Optuna-Spark/optuna-examples/databricks/init_optuna.sh +++ b/examples/ML+DL-Examples/Optuna-Spark/optuna-examples/databricks/init_optuna.sh @@ -55,7 +55,7 @@ fi # rapids import -SPARK_RAPIDS_VERSION=26.06.0 +SPARK_RAPIDS_VERSION=26.08.0 curl -L https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/${SPARK_RAPIDS_VERSION}/rapids-4-spark_2.12-${SPARK_RAPIDS_VERSION}.jar -o \ /databricks/jars/rapids-4-spark_2.12-${SPARK_RAPIDS_VERSION}.jar diff --git a/examples/ML+DL-Examples/Optuna-Spark/optuna-examples/databricks/start_cluster.sh b/examples/ML+DL-Examples/Optuna-Spark/optuna-examples/databricks/start_cluster.sh index 36d12c23c..3f1708789 100755 --- a/examples/ML+DL-Examples/Optuna-Spark/optuna-examples/databricks/start_cluster.sh +++ b/examples/ML+DL-Examples/Optuna-Spark/optuna-examples/databricks/start_cluster.sh @@ -26,7 +26,7 @@ json_config=$(cat < Local Disk). -*Note that the RAPIDS Accelerator for Apache Spark is not applicable in this case, since [multiple GPUs per executor are not yet supported](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#why-are-multiple-gpus-per-executor-not-supported). \ No newline at end of file +*Note that the NVIDIA cuDF plugin for Apache Spark is not applicable in this case, since [multiple GPUs per executor are not yet supported](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#why-are-multiple-gpus-per-executor-not-supported). \ No newline at end of file diff --git a/examples/ML+DL-Examples/Spark-Rapids-ML/pca/README.md b/examples/ML+DL-Examples/Spark-Rapids-ML/pca/README.md index c1cd8145f..3a9a32192 100644 --- a/examples/ML+DL-Examples/Spark-Rapids-ML/pca/README.md +++ b/examples/ML+DL-Examples/Spark-Rapids-ML/pca/README.md @@ -7,10 +7,10 @@ The notebook uses PCA to reduce a random dataset with 2048 feature dimensions to Please refer to the Spark-Rapids-ML [README](https://github.com/NVIDIA/spark-rapids-ml/blob/HEAD/python) to setup the RAPIDS conda environment and install Spark-Rapids-ML dependencies. -## Download RAPIDS Jar from Maven Central +## Download the NVIDIA cuDF plugin for Apache Spark jar from Maven Central -Download the [Spark-Rapids plugin](https://nvidia.github.io/spark-rapids/docs/download.html#download-rapids-accelerator-for-apache-spark-v26060). -For Spark-RAPIDS-ML version 26.06.0, download the RAPIDS jar from Maven Central: [rapids-4-spark_2.12-26.06.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar). +Download the [cuDF plugin](https://nvidia.github.io/cudf-spark/docs/download.html#download-rapids-accelerator-for-apache-spark-v26080). +For Spark-RAPIDS-ML version 26.08.0, download the cuDF plugin jar from Maven Central: [rapids-4-spark_2.12-26.08.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar). ## Running the Notebooks diff --git a/examples/ML+DL-Examples/Spark-Rapids-ML/pca/notebooks/pca.ipynb b/examples/ML+DL-Examples/Spark-Rapids-ML/pca/notebooks/pca.ipynb index a06a475c4..e4a245943 100644 --- a/examples/ML+DL-Examples/Spark-Rapids-ML/pca/notebooks/pca.ipynb +++ b/examples/ML+DL-Examples/Spark-Rapids-ML/pca/notebooks/pca.ipynb @@ -6,7 +6,7 @@ "source": [ "## Principal Component Analysis (PCA)\n", "\n", - "In this notebook, we will demonstrate the end-to-end workflow of Spark RAPIDS accelerated PCA." + "In this notebook, we will demonstrate the end-to-end workflow of PCA accelerated by the NVIDIA cuDF plugin for Apache Spark." ] }, { @@ -57,7 +57,7 @@ " import os\n", " import requests\n", "\n", - " SPARK_RAPIDS_VERSION = \"26.06.0\"\n", + " SPARK_RAPIDS_VERSION = \"26.08.0\"\n", " rapids_jar = f\"rapids-4-spark_2.12-{SPARK_RAPIDS_VERSION}.jar\"\n", " if not os.path.exists(rapids_jar):\n", " print(\"Downloading spark rapids jar\")\n", diff --git a/examples/SQL+DF-Examples/demo/Spark_get_json_object.ipynb b/examples/SQL+DF-Examples/demo/Spark_get_json_object.ipynb index 2231cb46a..c76a30858 100644 --- a/examples/SQL+DF-Examples/demo/Spark_get_json_object.ipynb +++ b/examples/SQL+DF-Examples/demo/Spark_get_json_object.ipynb @@ -7,7 +7,7 @@ "id": "Td_alkbOv3Aj" }, "source": [ - "# Spark RAPIDS get_json_object acceleration\n", + "# NVIDIA cuDF plugin for Apache Spark: get_json_object acceleration\n", "\n" ] }, @@ -41,7 +41,7 @@ "id": "ZfNDlz0SM0DB" }, "source": [ - "# Let's get started using the RAPIDS Accelerator for Apache Spark" + "# Let's get started using the cuDF plugin" ] }, { diff --git a/examples/SQL+DF-Examples/demo/Spark_parquet_microkernels.ipynb b/examples/SQL+DF-Examples/demo/Spark_parquet_microkernels.ipynb index ba27cf064..bf6dc9b60 100644 --- a/examples/SQL+DF-Examples/demo/Spark_parquet_microkernels.ipynb +++ b/examples/SQL+DF-Examples/demo/Spark_parquet_microkernels.ipynb @@ -16,7 +16,7 @@ " \"id\": \"Td_alkbOv3Aj\"\n", " },\n", " \"source\": [\n", - " \"# Spark RAPIDS Parquet acceleration\\n\",\n", + " \"# NVIDIA cuDF plugin for Apache Spark: Parquet acceleration\\n\",\n", " \"\\n\"\n", " ]\n", " },\n", @@ -50,7 +50,7 @@ " \"id\": \"ZfNDlz0SM0DB\"\n", " },\n", " \"source\": [\n", - " \"# Let's get started using the RAPIDS Accelerator for Apache Spark\"\n", + " \"# Let's get started using the cuDF plugin\"\n", " ]\n", " },\n", " {\n", diff --git a/examples/SQL+DF-Examples/micro-benchmarks/notebooks/micro-benchmarks-gpu.ipynb b/examples/SQL+DF-Examples/micro-benchmarks/notebooks/micro-benchmarks-gpu.ipynb index dc96881eb..b75cca01c 100644 --- a/examples/SQL+DF-Examples/micro-benchmarks/notebooks/micro-benchmarks-gpu.ipynb +++ b/examples/SQL+DF-Examples/micro-benchmarks/notebooks/micro-benchmarks-gpu.ipynb @@ -22,7 +22,7 @@ "import os\n", "# Change to your cluster ip:port and directories\n", "SPARK_MASTER_URL = os.getenv(\"SPARK_MASTER_URL\", \"spark:your-ip:port\")\n", - "RAPIDS_JAR = os.getenv(\"RAPIDS_JAR\", \"/your-path/rapids-4-spark_2.12-26.06.0.jar\")\n" + "RAPIDS_JAR = os.getenv(\"RAPIDS_JAR\", \"/your-path/rapids-4-spark_2.12-26.08.0.jar\")\n" ] }, { diff --git a/examples/SQL+DF-Examples/retail-analytics/README.md b/examples/SQL+DF-Examples/retail-analytics/README.md index d65dd2931..cba4f3f99 100644 --- a/examples/SQL+DF-Examples/retail-analytics/README.md +++ b/examples/SQL+DF-Examples/retail-analytics/README.md @@ -4,4 +4,4 @@ This repository contains two Jupyter notebooks: Data Generation: This notebook generates sample data that can be used for analysis. It demonstrates how to use various Python libraries to create synthetic data sets that can be used for testing and experimentation. This notebook can be run in GCP n1-standard-32 instance type -Data Cleaning and Analysis: This notebook takes the generated data and performs a series of cleaning and analysis tasks. It demonstrates how to use Spark RAPIDS library to manipulate and analyze data sets. +Data Cleaning and Analysis: This notebook takes the generated data and performs a series of cleaning and analysis tasks. It demonstrates how to use the NVIDIA cuDF plugin for Apache Spark to manipulate and analyze data sets. diff --git a/examples/SQL+DF-Examples/tpcds/README.md b/examples/SQL+DF-Examples/tpcds/README.md index c365b663c..bb6003898 100644 --- a/examples/SQL+DF-Examples/tpcds/README.md +++ b/examples/SQL+DF-Examples/tpcds/README.md @@ -1,4 +1,4 @@ -# TPC-DS Scale Factor 10 (GiB) - CPU Spark vs GPU Spark +# TPC-DS Scale Factor 10 (GiB) - CPU Spark vs NVIDIA cuDF plugin for Apache Spark [TPC-DS](https://www.tpc.org/tpcds/) is a decision support benchmark often used to evaluate performance of OLAP Databases and Big Data systems. @@ -7,7 +7,7 @@ The notebook in this folder runs a user-specified subset of the TPC-DS queries o Scale Factor 10 (GiB) dataset. It uses [TPCDS PySpark](https://github.com/cerndb/SparkTraining/blob/master/notebooks/TPCDS_PySpark_CERN_SWAN_getstarted.ipynb) to execute TPC-DS queries with SparkSQL on GPU and CPU capturing the metrics as a Pandas dataframe. It then plots a comparison bar chart visualizing -the GPU acceleration achieved for the queries run with RAPIDS Spark in this +the GPU acceleration achieved for the queries run with the cuDF plugin in this very notebook. This notebook can be opened and executed using standard @@ -23,6 +23,6 @@ Google Colab and connect it to a [GPU instance](https://research.google.com/cola Here is the bar chart from a recent execution on Google Colab's T4 High RAM instance using -RAPIDS Spark 26.06.0 with Apache Spark 3.5.0 +the cuDF plugin 26.08.0 with Apache Spark 3.5.0 ![tpcds-speedup](/docs/img/guides/tpcds.png) diff --git a/examples/SQL+DF-Examples/tpcds/notebooks/TPCDS-SF10.ipynb b/examples/SQL+DF-Examples/tpcds/notebooks/TPCDS-SF10.ipynb index 11e2332e0..817325f90 100644 --- a/examples/SQL+DF-Examples/tpcds/notebooks/TPCDS-SF10.ipynb +++ b/examples/SQL+DF-Examples/tpcds/notebooks/TPCDS-SF10.ipynb @@ -11,7 +11,7 @@ "tags": [] }, "source": [ - "# TPC-DS 10GiB - Apache Spark acceleration on GPU with RAPIDS Spark\n", + "# TPC-DS 10GiB - Apache Spark acceleration on GPU with the NVIDIA cuDF plugin for Apache Spark\n", "\n", "based on https://colab.research.google.com/github/LucaCanali/Miscellaneous/blob/master/Performance_Testing/TPCDS_PySpark/Labs_and_Notes/TPCDS_PySpark_getstarted.ipynb#scrollTo=6bab7772" ] @@ -30,7 +30,7 @@ "outputs": [], "source": [ "spark_version='3.5.5'\n", - "rapids_version='26.06.0'\n", + "rapids_version='26.08.0'\n", "sparkmeasure_version='0.27'" ] }, @@ -134,7 +134,7 @@ "id": "6tgF9LWcgUEs" }, "source": [ - "# Init a SparkSession with RAPIDS Spark" + "# Initialize a SparkSession with the cuDF plugin" ] }, { diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/README.md b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/README.md index 5ceb8d94d..715de1f31 100644 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/README.md +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/README.md @@ -1,12 +1,12 @@ -# RAPIDS Accelerated UDF Examples +# NVIDIA cuDF plugin for Apache Spark UDF Examples -This project contains sample implementations of RAPIDS accelerated user-defined functions. +This project contains sample implementations of GPU-accelerated user-defined functions for the cuDF plugin. The ideal solution would be to replace the UDF with a series of DataFrame or SQL operations. If that is not possible, we also provide -a [UDF compiler extension](https://nvidia.github.io/spark-rapids/docs/additional-functionality/udf-to-catalyst-expressions.html) +a [UDF compiler extension](https://nvidia.github.io/cudf-spark/docs/additional-functionality/udf-to-catalyst-expressions.html) to translate UDFs to Catalyst expressions. The extension is limited to only support compiling simple -operations. For complicated cases, you can choose to implement a RAPIDS accelerated UDF. +operations. For complicated cases, you can choose to implement a GPU-accelerated UDF for the cuDF plugin. ## Spark Scala UDF Examples @@ -23,7 +23,7 @@ implement via the Java API is ease of development, but the memory model is not f GPU operations because the JVM makes the assumption that everything we're trying to do is in heap memory. We need to free the GPU resources in a timely manner with try-finally blocks. Note that we need to implement both CPU and GPU functions so the UDF will still work if a higher-level operation -involving the RAPIDS accelerated UDF falls back to the CPU. +involving the GPU-accelerated UDF for the cuDF plugin falls back to the CPU. - [URLDecode](src/main/scala/com/nvidia/spark/rapids/udf/scala/URLDecode.scala) decodes URL-encoded strings using the @@ -34,7 +34,7 @@ involving the RAPIDS accelerated UDF falls back to the CPU. ## Spark Java UDF Examples -Below are some examples for implementing RAPIDS accelerated Scala UDF via JNI and native code. If +Below are some examples for implementing GPU-accelerated Scala UDFs for the cuDF plugin via JNI and native code. If there is no existing simple Java API we could leverage, we can write native custom code. Take [CosineSimilarity](src/main/java/com/nvidia/spark/rapids/udf/java/CosineSimilarity.java) as the example, the Java class for the UDF is similar as the previous URLDecode/URLEncode demo. We need to @@ -48,7 +48,7 @@ we can leverage the [Thrust template library](https://docs.nvidia.com/cuda/thrus write the standard algorithms for GPU parallelizing code. The benefit of implementing the UDF in native code is for maximum control over GPU memory utilization and performance. However the trade-off is a more complicated build environment, as we need to build against libcudf with -significantly longer build times. Implementing a RAPIDS accelerated UDF in native code is a +significantly longer build times. Implementing a GPU-accelerated UDF for the cuDF plugin in native code is a significant effort. - [URLDecode](src/main/java/com/nvidia/spark/rapids/udf/java/URLDecode.java) @@ -63,7 +63,7 @@ significant effort. ## Hive UDF Examples -Below are some examples for implementing RAPIDS accelerated Hive UDF via JNI and native code. +Below are some examples for implementing GPU-accelerated Hive UDFs for the cuDF plugin via JNI and native code. - [URLDecode](src/main/java/com/nvidia/spark/rapids/udf/hive/URLDecode.java) implements a Hive simple UDF using the @@ -274,7 +274,7 @@ then do the following inside the Docker container. ### Get jars from Maven Central -[rapids-4-spark_2.12-26.06.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar) +[rapids-4-spark_2.12-26.08.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar) ### Launch a local mode Spark diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/extract-cudf-libs.sh b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/extract-cudf-libs.sh index 79f78cbab..d9f56d298 100755 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/extract-cudf-libs.sh +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/extract-cudf-libs.sh @@ -31,10 +31,10 @@ # RAPIDS4SPARK_VERSION - rapids-4-spark version (e.g., 26.08.0 or 26.08.0-SNAPSHOT) # SCALA_VERSION - Scala binary version (e.g., 2.12, 2.13) # CUDA_VERSION - CUDA version (e.g., cuda11, cuda12) -# CUDF_BRANCH - cuDF git branch for headers (e.g., main, branch-26.06) +# CUDF_BRANCH - cuDF git branch for headers (e.g., main, branch-26.08) # # Example with overrides: -# RAPIDS4SPARK_VERSION=26.06.0 CUDA_VERSION=cuda11 ./extract-cudf-libs.sh +# RAPIDS4SPARK_VERSION=26.08.0 CUDA_VERSION=cuda11 ./extract-cudf-libs.sh ############################################################################### set -e @@ -126,7 +126,7 @@ else echo " $JAR_PATH_NO_CLASSIFIER" echo "" echo "For SNAPSHOT versions:" - echo " cd /path/to/spark-rapids" + echo " cd /path/to/cudf-spark" echo " mvn clean install -DskipTests" echo "" echo "For release versions:" diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/pom.xml b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/pom.xml index 8666dcecf..7c547e240 100644 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/pom.xml +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/pom.xml @@ -20,10 +20,9 @@ 4.0.0 com.nvidia rapids-4-spark-udf-examples_2.12 - RAPIDS Accelerator for Apache Spark UDF Examples - Sample implementations of RAPIDS accelerated - user defined functions for use with the RAPIDS Accelerator - for Apache Spark + NVIDIA cuDF plugin for Apache Spark UDF Examples + Sample implementations of GPU-accelerated user-defined functions + for use with the NVIDIA cuDF plugin for Apache Spark 26.08.0-SNAPSHOT @@ -37,7 +36,7 @@ cuda12 2.12 - 26.06.0 + 26.08.0 3.1.1 2.12.15 ${project.build.directory}/cpp-build @@ -370,8 +369,8 @@ - - + + diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/cpp/CMakeLists.txt b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/cpp/CMakeLists.txt index f524d5bf5..76949b0f3 100755 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/cpp/CMakeLists.txt +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/cpp/CMakeLists.txt @@ -141,7 +141,7 @@ if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/../../../target/USE_SOURCE_BUILD") " to building from source\n" "\n" " 2. Build and install rapids-4-spark:\n" - " cd /path/to/spark-rapids\n" + " cd /path/to/cudf-spark\n" " mvn clean install -DskipTests\n" "\n" " 3. Explicitly use source build:\n" @@ -202,7 +202,7 @@ if(USE_PREBUILT_CUDF AND NOT USER_SET_USE_PREBUILT_CUDF) "\n" "To use fast build mode in future:\n" " 1. For SNAPSHOT versions: Build and install rapids-4-spark\n" - " cd /path/to/spark-rapids\n" + " cd /path/to/cudf-spark\n" " mvn clean install -DskipTests\n" " 2. Run: mvn clean package -Pudf-native-examples\n" "\n" diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/hive/URLDecode.java b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/hive/URLDecode.java index e97f0a21c..08978001b 100644 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/hive/URLDecode.java +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/hive/URLDecode.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 2020-2022, NVIDIA CORPORATION. + * Copyright (c) 2020-2026, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -29,7 +29,7 @@ * A Hive user-defined function (UDF) that decodes URL-encoded strings. * This class demonstrates how to implement a simple Hive UDF that also * provides a RAPIDS implementation that can run on the GPU when the query - * is executed with the RAPIDS Accelerator for Apache Spark. + * is executed with the NVIDIA cuDF plugin for Apache Spark. */ public class URLDecode extends UDF implements RapidsUDF { diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/hive/URLEncode.java b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/hive/URLEncode.java index 98bcb73eb..86622d015 100644 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/hive/URLEncode.java +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/hive/URLEncode.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 2020-2022, NVIDIA CORPORATION. + * Copyright (c) 2020-2026, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -35,7 +35,7 @@ * A Hive user-defined function (UDF) that URL-encodes strings. * This class demonstrates how to implement a Hive GenericUDF that also * provides a RAPIDS implementation that can run on the GPU when the query - * is executed with the RAPIDS Accelerator for Apache Spark. + * is executed with the NVIDIA cuDF plugin for Apache Spark. */ public class URLEncode extends GenericUDF implements RapidsUDF { private transient PrimitiveObjectInspectorConverter.TextConverter converter; diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/java/URLDecode.java b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/java/URLDecode.java index 96d07384f..11be8b1ca 100644 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/java/URLDecode.java +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/java/URLDecode.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 2021-2022, NVIDIA CORPORATION. + * Copyright (c) 2021-2026, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -29,7 +29,7 @@ * A Java user-defined function (UDF) that decodes URL-encoded strings. * This class demonstrates how to implement a Java UDF that also * provides a RAPIDS implementation that can run on the GPU when the query - * is executed with the RAPIDS Accelerator for Apache Spark. + * is executed with the NVIDIA cuDF plugin for Apache Spark. */ public class URLDecode implements UDF1, RapidsUDF { /** Row-by-row implementation that executes on the CPU */ diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/java/URLEncode.java b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/java/URLEncode.java index 13bdfff55..f7becb969 100644 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/java/URLEncode.java +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/java/com/nvidia/spark/rapids/udf/java/URLEncode.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 2021-2022, NVIDIA CORPORATION. + * Copyright (c) 2021-2026, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -28,7 +28,7 @@ * A Java user-defined function (UDF) that URL-encodes strings. * This class demonstrates how to implement a Java UDF that also * provides a RAPIDS implementation that can run on the GPU when the query - * is executed with the RAPIDS Accelerator for Apache Spark. + * is executed with the NVIDIA cuDF plugin for Apache Spark. */ public class URLEncode implements UDF1, RapidsUDF { /** Row-by-row implementation that executes on the CPU */ diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/scala/com/nvidia/spark/rapids/udf/scala/URLDecode.scala b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/scala/com/nvidia/spark/rapids/udf/scala/URLDecode.scala index 5cf8da123..4c0ab0f00 100644 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/scala/com/nvidia/spark/rapids/udf/scala/URLDecode.scala +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/scala/com/nvidia/spark/rapids/udf/scala/URLDecode.scala @@ -1,5 +1,5 @@ /* - * Copyright (c) 2021-2022, NVIDIA CORPORATION. + * Copyright (c) 2021-2026, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -25,7 +25,7 @@ import com.nvidia.spark.RapidsUDF * A Scala user-defined function (UDF) that decodes URL-encoded strings. * This class demonstrates how to implement a Scala UDF that also * provides a RAPIDS implementation that can run on the GPU when the query - * is executed with the RAPIDS Accelerator for Apache Spark. + * is executed with the NVIDIA cuDF plugin for Apache Spark. */ class URLDecode extends Function[String, String] with RapidsUDF with Serializable { /** Row-by-row implementation that executes on the CPU */ diff --git a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/scala/com/nvidia/spark/rapids/udf/scala/URLEncode.scala b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/scala/com/nvidia/spark/rapids/udf/scala/URLEncode.scala index 6facee3b9..d9fda5769 100644 --- a/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/scala/com/nvidia/spark/rapids/udf/scala/URLEncode.scala +++ b/examples/UDF-Examples/RAPIDS-accelerated-UDFs/src/main/scala/com/nvidia/spark/rapids/udf/scala/URLEncode.scala @@ -1,5 +1,5 @@ /* - * Copyright (c) 2021-2022, NVIDIA CORPORATION. + * Copyright (c) 2021-2026, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -25,7 +25,7 @@ import com.nvidia.spark.RapidsUDF * A Scala user-defined function (UDF) that URL-encodes strings. * This class demonstrates how to implement a Scala UDF that also * provides a RAPIDS implementation that can run on the GPU when the query - * is executed with the RAPIDS Accelerator for Apache Spark. + * is executed with the NVIDIA cuDF plugin for Apache Spark. */ class URLEncode extends Function[String, String] with RapidsUDF with Serializable { /** Row-by-row implementation that executes on the CPU */ diff --git a/examples/XGBoost-Examples/README.md b/examples/XGBoost-Examples/README.md index 7ec7dda9e..91b660372 100644 --- a/examples/XGBoost-Examples/README.md +++ b/examples/XGBoost-Examples/README.md @@ -5,8 +5,8 @@ The Scala based XGBoost examples here use [DMLC’s version](https://repo1.maven The pyspark based XGBoost examples requires [installing RAPIDS via pip](https://rapids.ai/pip.html#install). Most data scientists spend a lot of time not only on Training models but also processing the large amounts of data needed to train these models. -As you can see below, Pyspark+XGBoost training on GPUs can be up to 13X and data processing using -RAPIDS Accelerator can also be accelerated with an end-to-end speed-up of 11X on GPU compared to CPU. +As shown below, PySpark and XGBoost training on GPUs can be up to 13X faster, while data processing with +the NVIDIA cuDF plugin for Apache Spark can deliver an end-to-end speedup of 11X over CPU execution. In the public cloud, better performance can lead to significantly lower costs as demonstrated in this [blog](https://developer.nvidia.com/blog/gpu-accelerated-spark-xgboost/). ![mortgage-speedup](/docs/img/guides/mortgage-perf.png) @@ -16,7 +16,7 @@ with a 8 A100 GPU and 1024 CPU vcores cluster, the performance is affected by ma including data size and type of GPU. In this folder, there are three blue prints for users to learn about using -Spark XGBoost and RAPIDS Accelerator on GPUs : +Spark XGBoost and the cuDF plugin on GPUs: 1. Mortgage Prediction 2. Agaricus Classification diff --git a/examples/XGBoost-Examples/mortgage/notebooks/python/MortgageETL+XGBoost.ipynb b/examples/XGBoost-Examples/mortgage/notebooks/python/MortgageETL+XGBoost.ipynb index d90fbb156..bc28015f3 100644 --- a/examples/XGBoost-Examples/mortgage/notebooks/python/MortgageETL+XGBoost.ipynb +++ b/examples/XGBoost-Examples/mortgage/notebooks/python/MortgageETL+XGBoost.ipynb @@ -10,7 +10,7 @@ "\n", "# ETL + XGBoost train & transform\n", "\n", - "This notebook is an end-to-end example of ETL + XGBoost Train & Transform by using [Spark-Rapids](https://github.com/NVIDIA/spark-rapids) and [XGBoost](https://github.com/dmlc/xgboost) with GPU accelerated.\n", + "This notebook is an end-to-end example of ETL + XGBoost Train & Transform using the [NVIDIA cuDF plugin for Apache Spark](https://github.com/NVIDIA/cudf-spark) and [XGBoost](https://github.com/dmlc/xgboost) with GPU acceleration.\n", "
The main steps:\n", "1. Run ETL to generate 2 datasets for train and test
\n", " You can choose to save the datasets or not by setting \"is_save_dataset\" to True or False.
\n", diff --git a/examples/XGBoost-Examples/mortgage/notebooks/python/MortgageETL.ipynb b/examples/XGBoost-Examples/mortgage/notebooks/python/MortgageETL.ipynb index c50149ab4..020325c08 100644 --- a/examples/XGBoost-Examples/mortgage/notebooks/python/MortgageETL.ipynb +++ b/examples/XGBoost-Examples/mortgage/notebooks/python/MortgageETL.ipynb @@ -9,7 +9,7 @@ "Dataset is derived from Fannie Mae’s [Single-Family Loan Performance Data](http://www.fanniemae.com/portal/funding-the-market/data/loan-performance-data.html) with all rights reserved by Fannie Mae. Refer to these [instructions](https://github.com/NVIDIA/spark-rapids-examples/blob/branch-24.12/docs/get-started/xgboost-examples/dataset/mortgage.md) to download the dataset.\n", "\n", "### 2. Download needed jars\n", - "* [rapids-4-spark_2.12-26.06.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar)\n", + "* [rapids-4-spark_2.12-26.08.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar)\n", "\n", "\n", "### 3. Start Spark Standalone\n", @@ -17,7 +17,7 @@ "\n", "### 4. Add ENV\n", "```\n", - "$ export SPARK_JARS=rapids-4-spark_2.12-26.06.0.jar\n", + "$ export SPARK_JARS=rapids-4-spark_2.12-26.08.0.jar\n", "$ export PYSPARK_DRIVER_PYTHON=jupyter \n", "$ export PYSPARK_DRIVER_PYTHON_OPTS=notebook\n", "```\n", diff --git a/examples/XGBoost-Examples/mortgage/notebooks/python/mortgage-gpu.ipynb b/examples/XGBoost-Examples/mortgage/notebooks/python/mortgage-gpu.ipynb index 44889501a..16e3971f6 100644 --- a/examples/XGBoost-Examples/mortgage/notebooks/python/mortgage-gpu.ipynb +++ b/examples/XGBoost-Examples/mortgage/notebooks/python/mortgage-gpu.ipynb @@ -6,7 +6,7 @@ "source": [ "# Introduction to XGBoost Spark with GPU\n", "\n", - "The goal of this notebook is to show how to train a XGBoost Model with Spark RAPIDS XGBoost library on GPUs. The dataset used with this notebook is derived from Fannie Mae’s Single-Family Loan Performance Data with all rights reserved by Fannie Mae. This processed dataset is redistributed with permission and consent from Fannie Mae. This notebook uses XGBoost to train 12-month mortgage loan delinquency prediction model .\n", + "The goal of this notebook is to show how to train a XGBoost Model with the XGBoost4J-Spark library on GPUs. The dataset used with this notebook is derived from Fannie Mae’s Single-Family Loan Performance Data with all rights reserved by Fannie Mae. This processed dataset is redistributed with permission and consent from Fannie Mae. This notebook uses XGBoost to train 12-month mortgage loan delinquency prediction model .\n", "\n", "A few libraries required for this notebook:\n", " 1. cudf-cu11\n", diff --git a/examples/XGBoost-Examples/mortgage/notebooks/scala/mortgage-ETL.ipynb b/examples/XGBoost-Examples/mortgage/notebooks/scala/mortgage-ETL.ipynb index c98a99560..956937ea9 100644 --- a/examples/XGBoost-Examples/mortgage/notebooks/scala/mortgage-ETL.ipynb +++ b/examples/XGBoost-Examples/mortgage/notebooks/scala/mortgage-ETL.ipynb @@ -20,14 +20,14 @@ "Refer to these [instructions](https://github.com/NVIDIA/spark-rapids-examples/blob/branch-23.12/docs/get-started/xgboost-examples/dataset/mortgage.md) to download the dataset.\n", "\n", "### 2. Download needed jars\n", - "* [rapids-4-spark_2.12-26.06.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar)\n", + "* [rapids-4-spark_2.12-26.08.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar)\n", "\n", "### 3. Start Spark Standalone\n", "Before Running the script, please setup Spark standalone mode\n", "\n", "### 4. Add ENV\n", "```\n", - "$ export SPARK_JARS=rapids-4-spark_2.12-26.06.0.jar\n", + "$ export SPARK_JARS=rapids-4-spark_2.12-26.08.0.jar\n", "\n", "```\n", "\n", diff --git a/examples/XGBoost-Examples/mortgage/notebooks/scala/mortgage-gpu.ipynb b/examples/XGBoost-Examples/mortgage/notebooks/scala/mortgage-gpu.ipynb index df0c1f067..f2f229a57 100644 --- a/examples/XGBoost-Examples/mortgage/notebooks/scala/mortgage-gpu.ipynb +++ b/examples/XGBoost-Examples/mortgage/notebooks/scala/mortgage-gpu.ipynb @@ -6,7 +6,7 @@ "source": [ "# Introduction to XGBoost Spark with GPU\n", "\n", - "The goal of this notebook is to show how to train a XGBoost Model with Spark RAPIDS XGBoost library on GPUs. The dataset used with this notebook is derived from Fannie Mae’s Single-Family Loan Performance Data with all rights reserved by Fannie Mae. This processed dataset is redistributed with permission and consent from Fannie Mae. This notebook uses XGBoost to train 12-month mortgage loan delinquency prediction model.\n", + "The goal of this notebook is to show how to train a XGBoost Model with the XGBoost4J-Spark library on GPUs. The dataset used with this notebook is derived from Fannie Mae’s Single-Family Loan Performance Data with all rights reserved by Fannie Mae. This processed dataset is redistributed with permission and consent from Fannie Mae. This notebook uses XGBoost to train 12-month mortgage loan delinquency prediction model.\n", "\n", "## Load libraries\n", "First load some common libraries will be used by both GPU version and CPU version xgboost." diff --git a/examples/XGBoost-Examples/taxi/notebooks/python/taxi-ETL.ipynb b/examples/XGBoost-Examples/taxi/notebooks/python/taxi-ETL.ipynb index f4e6b6df2..cf96a96d0 100644 --- a/examples/XGBoost-Examples/taxi/notebooks/python/taxi-ETL.ipynb +++ b/examples/XGBoost-Examples/taxi/notebooks/python/taxi-ETL.ipynb @@ -19,14 +19,14 @@ "All data could be found at https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page\n", "\n", "### 2. Download needed jars\n", - "* [rapids-4-spark_2.12-26.06.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar)\n", + "* [rapids-4-spark_2.12-26.08.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar)\n", "\n", "### 3. Start Spark Standalone\n", "Before running the script, please setup Spark standalone mode\n", "\n", "### 4. Add ENV\n", "```\n", - "$ export SPARK_JARS=rapids-4-spark_2.12-26.06.0.jar\n", + "$ export SPARK_JARS=rapids-4-spark_2.12-26.08.0.jar\n", "$ export PYSPARK_DRIVER_PYTHON=jupyter \n", "$ export PYSPARK_DRIVER_PYTHON_OPTS=notebook\n", "```\n", diff --git a/examples/XGBoost-Examples/taxi/notebooks/scala/taxi-ETL.ipynb b/examples/XGBoost-Examples/taxi/notebooks/scala/taxi-ETL.ipynb index 3091a194f..edb9e49c5 100644 --- a/examples/XGBoost-Examples/taxi/notebooks/scala/taxi-ETL.ipynb +++ b/examples/XGBoost-Examples/taxi/notebooks/scala/taxi-ETL.ipynb @@ -19,14 +19,14 @@ "All data could be found at https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page\n", "\n", "### 2. Download needed jar\n", - "* [rapids-4-spark_2.12-26.06.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.06.0/rapids-4-spark_2.12-26.06.0.jar)\n", + "* [rapids-4-spark_2.12-26.08.0.jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/26.08.0/rapids-4-spark_2.12-26.08.0.jar)\n", "\n", "### 3. Start Spark Standalone\n", "Before running the script, please setup Spark standalone mode\n", "\n", "### 4. Add ENV\n", "```\n", - "$ export SPARK_JARS=rapids-4-spark_2.12-26.06.0.jar\n", + "$ export SPARK_JARS=rapids-4-spark_2.12-26.08.0.jar\n", "\n", "```\n", "\n", diff --git a/examples/spark-connect-gpu/client/README.md b/examples/spark-connect-gpu/client/README.md index 892a723be..5c0e6eaa2 100644 --- a/examples/spark-connect-gpu/client/README.md +++ b/examples/spark-connect-gpu/client/README.md @@ -1,7 +1,7 @@ # GPU-Accelerated Spark Connect for ETL and ML (Spark 4.0) This project demonstrates some python/scala batch jobs and a complete GPU-accelerated ETL and -Machine Learning pipeline using Apache Spark 4.0 with Spark Connect, featuring the RAPIDS Accelerator. +Machine Learning pipeline using Apache Spark 4.0 with Spark Connect, featuring the NVIDIA cuDF plugin for Apache Spark. ## 🏗️ Architecture @@ -71,7 +71,7 @@ found [here](../server/README.md). Most users won't need to adjust the GPU configurations. However, if you'd like to tune your GPU for better performance, refer to the -[advanced GPU configurations documentation](https://nvidia.github.io/spark-rapids/docs/additional-functionality/advanced_configs.html). +[advanced GPU configurations documentation](https://nvidia.github.io/cudf-spark/docs/additional-functionality/advanced_configs.html). **Note**: Configurations prefixed with spark.rapids.sql are session-specific and can be set safely. However, those marked as **startup** will not take @@ -107,5 +107,5 @@ docker logs spark-connect-client - [Apache Spark 4.0 Documentation](https://spark.apache.org/docs/latest/) - [Spark Connect Guide](https://spark.apache.org/docs/latest/spark-connect-overview.html) -- [NVIDIA RAPIDS Accelerator](https://nvidia.github.io/spark-rapids/) +- [NVIDIA cuDF plugin for Apache Spark](https://nvidia.github.io/cudf-spark/) - [Data and AI Summit Session](https://www.databricks.com/dataaisummit/session/gpu-accelerated-spark-connect) diff --git a/examples/spark-connect-gpu/server/README.md b/examples/spark-connect-gpu/server/README.md index f1a8d850e..cf562e0a1 100644 --- a/examples/spark-connect-gpu/server/README.md +++ b/examples/spark-connect-gpu/server/README.md @@ -1,12 +1,12 @@ # GPU-Accelerated Spark Connect Server This project demonstrates how to set up a GPU-accelerated Spark server using Apache Spark 4.0 -with Spark Connect, featuring the RAPIDS Accelerator. +with Spark Connect, featuring the NVIDIA cuDF plugin for Apache Spark. ## 🚀 Key Features - **Apache Spark 4.0** with cutting-edge Spark Connect capabilities -- **GPU acceleration** via RAPIDS Accelerator +- **GPU acceleration** via the cuDF plugin - **MLlib over Spark Connect** - new in Spark 4.0 - **Zero-code-change acceleration** - existing Spark applications automatically benefit - **Jupyter Lab integration** for interactive development @@ -25,7 +25,7 @@ not have GPU capability service requiring and having access to the host GPUs ### Middle Tier -3. **Spark Connect Server** (`spark-connect-server`) - gRPC interface with the RAPIDS integration +3. **Spark Connect Server** (`spark-connect-server`) - gRPC interface with the cuDF plugin integration ### Proxy Service 4. nginx configured as provide access to various Apache Spark WebUI using the Docker network @@ -149,8 +149,8 @@ path. Otherwise, we use variables starting with `local_`. - **Features**: GPU resource discovery and task execution ### Spark Connect Server -- **Image**: Custom build based on `apache/spark:4.0.0` with Spark RAPIDS ETL and ML Plugins -- **RAPIDS Version**: 26.06.0 for CUDA 12 +- **Image**: Custom build based on `apache/spark:4.0.0` with the cuDF plugin for ETL and Spark-Rapids-ML for ML +- **cuDF plugin version**: 26.08.0 for CUDA 12 - **Ports**: 15002 (gRPC), 4040 (Driver UI) - **Configuration**: Optimized for GPU acceleration with memory management @@ -185,5 +185,5 @@ Spark executor logs can be accessed via the Spark UI as usual. - [Apache Spark 4.0 Documentation](https://spark.apache.org/docs/latest/) - [Spark Connect Guide](https://spark.apache.org/docs/latest/spark-connect-overview.html) -- [NVIDIA RAPIDS Accelerator](https://nvidia.github.io/spark-rapids/) +- [NVIDIA cuDF plugin for Apache Spark](https://nvidia.github.io/cudf-spark/) - [Data and AI Summit Session](https://www.databricks.com/dataaisummit/session/gpu-accelerated-spark-connect) diff --git a/examples/spark-connect-gpu/server/docker-compose.yaml b/examples/spark-connect-gpu/server/docker-compose.yaml index 22f8b7b0c..9bb6d47b6 100644 --- a/examples/spark-connect-gpu/server/docker-compose.yaml +++ b/examples/spark-connect-gpu/server/docker-compose.yaml @@ -74,7 +74,7 @@ services: dockerfile: Dockerfile args: - CUDA_VERSION=${CUDA_VERSION:-12} - - RAPIDS_VERSION=${RAPIDS_VERSION:-26.06.0} + - RAPIDS_VERSION=${RAPIDS_VERSION:-26.08.0} - REPO_URL=${REPO_URL:-https://repo1.maven.org/maven2} container_name: spark-connect-server hostname: spark-connect-server diff --git a/scripts/csp-startup-scripts/README.md b/scripts/csp-startup-scripts/README.md index 46b750019..6ad7f37e4 100644 --- a/scripts/csp-startup-scripts/README.md +++ b/scripts/csp-startup-scripts/README.md @@ -1,3 +1,3 @@ -# Startup Scripts for CSPs with Spark Rapids +# Startup Scripts for CSPs with the NVIDIA cuDF plugin for Apache Spark -With the exception of Dataproc, CSP offerings like EMR have specific set of steps that are required to enable the Spark Rapids Plugin in their environment. The set of scripts here automate parts of that process, for EMR currently. The exact usage can be found in our docs [here](https://docs.nvidia.com/spark-rapids/user-guide/latest/getting-started/aws-emr.html) +With the exception of Dataproc, CSP offerings like EMR have specific set of steps that are required to enable the cuDF plugin in their environment. The set of scripts here automate parts of that process, for EMR currently. The exact usage can be found in our docs [here](https://docs.nvidia.com/spark-rapids/user-guide/latest/getting-started/aws-emr.html) diff --git a/tools/databricks/README.md b/tools/databricks/README.md index 54e4bae2a..91198c041 100644 --- a/tools/databricks/README.md +++ b/tools/databricks/README.md @@ -1,6 +1,6 @@ # Databricks Qualification/Profiling Quick Start Notebooks -The RAPIDS Accelerator for Apache Spark includes two key tools for understanding the benefits of +The NVIDIA cuDF plugin for Apache Spark provides two key tools for understanding the benefits of GPU acceleration as well as analyzing GPU Spark jobs. For customers on Databricks, the quick start notebooks offer a simple interface for running the tools given a set of Spark event logs from CPU (qualification) or GPU (profiling) application runs. @@ -19,4 +19,4 @@ top of the notebook. After that, select *Run all* to execute the tools for the 1. Multiple event logs must be comma-separated. - For example: `/dbfs/path/to/eventlog1,/dbfs/path/to/eventlog2` -**Latest Tools Version Supported** 26.06.0 \ No newline at end of file +**Latest Tools Version Supported** 26.08.0 \ No newline at end of file diff --git a/tools/databricks/[RAPIDS Accelerator for Apache Spark] Profiling Tool Notebook Template.ipynb b/tools/databricks/[NVIDIA cuDF plugin for Apache Spark] Profiling Tool Notebook Template.ipynb similarity index 99% rename from tools/databricks/[RAPIDS Accelerator for Apache Spark] Profiling Tool Notebook Template.ipynb rename to tools/databricks/[NVIDIA cuDF plugin for Apache Spark] Profiling Tool Notebook Template.ipynb index f99221b8e..8bbe6ac2d 100644 --- a/tools/databricks/[RAPIDS Accelerator for Apache Spark] Profiling Tool Notebook Template.ipynb +++ b/tools/databricks/[NVIDIA cuDF plugin for Apache Spark] Profiling Tool Notebook Template.ipynb @@ -21,7 +21,7 @@ " }\n", " },\n", " \"source\": [\n", - " \"## Welcome to the Profiling Tool for the RAPIDS Accelerator for Apache Spark\\n\",\n", + " \"## Welcome to the Profiling Tool for the NVIDIA cuDF plugin for Apache Spark\\n\",\n", " \"\\n\",\n", " \"To run the profiling tool, enter the log path that represents the DBFS location of your Spark GPU event logs. Then, select \\\"Run all\\\" to execute the notebook. Once the notebook completes, various output tables will appear below. For more options on running the profiling tool, please refer to the [Profiling Tool User Guide](https://docs.nvidia.com/spark-rapids/user-guide/latest/profiling/quickstart.html#running-the-tool).\\n\",\n", " \"\\n\",\n", @@ -515,7 +515,7 @@ " }\n", " ]\n", " },\n", - " \"notebookName\": \"[RAPIDS Accelerator for Apache Spark] Profiling Tool Notebook Template\",\n", + " \"notebookName\": \"[NVIDIA cuDF plugin for Apache Spark] Profiling Tool Notebook Template\",\n", " \"widgets\": {\n", " \"Eventlog Path\": {\n", " \"currentValue\": \"/dbfs/user1/profiling_logs\",\n", diff --git a/tools/databricks/[RAPIDS Accelerator for Apache Spark] Qualification Tool Notebook Template.ipynb b/tools/databricks/[NVIDIA cuDF plugin for Apache Spark] Qualification Tool Notebook Template.ipynb similarity index 98% rename from tools/databricks/[RAPIDS Accelerator for Apache Spark] Qualification Tool Notebook Template.ipynb rename to tools/databricks/[NVIDIA cuDF plugin for Apache Spark] Qualification Tool Notebook Template.ipynb index f1d7ebe5d..e08518955 100644 --- a/tools/databricks/[RAPIDS Accelerator for Apache Spark] Qualification Tool Notebook Template.ipynb +++ b/tools/databricks/[NVIDIA cuDF plugin for Apache Spark] Qualification Tool Notebook Template.ipynb @@ -21,7 +21,7 @@ " }\n", " },\n", " \"source\": [\n", - " \"## Welcome to the Qualification Tool for the RAPIDS Accelerator for Apache Spark\\n\",\n", + " \"## Welcome to the Qualification Tool for the NVIDIA cuDF plugin for Apache Spark\\n\",\n", " \"\\n\",\n", " \"To run the qualification tool, enter the log path that represents the DBFS location of your Spark GPU event logs. Then, select \\\"Run all\\\" to execute the notebook. Once the notebook completes, various output tables will appear below. For more options on running the profiling tool, please refer to the [Qualification Tool User Guide](https://docs.nvidia.com/spark-rapids/user-guide/latest/qualification/quickstart.html#running-the-tool).\\n\",\n", " \"\\n\",\n", @@ -560,7 +560,7 @@ " \"2. Stage ID\\n\",\n", " \"3. Average Speedup Factor: The average estimated speed-up of all the operators in the given stage.\\n\",\n", " \"4. Stage Task Duration: The amount of time spent in tasks of SQL DataFrame operations for the given stage.\\n\",\n", - " \"5. Unsupported Task Duration: The sum of task durations for the unsupported operators. For more details, see [Supported Operators](https://nvidia.github.io/spark-rapids/docs/supported_ops.html).\\n\",\n", + " \"5. Unsupported Task Duration: The sum of task durations for the unsupported operators. For more details, see [Supported Operators](https://nvidia.github.io/cudf-spark/docs/supported_ops.html).\\n\",\n", " \"6. Stage Estimated: Indicates if the stage duration had to be estimated (True or False).\\n\"\n", " ]\n", " },\n", @@ -605,7 +605,7 @@ " \"source\": [\n", " \"## Execs Output\\n\",\n", " \"\\n\",\n", - " \"The Qualification tool generates a report of the “Exec” in the “SparkPlan” or “Executor Nodes” along with the estimated acceleration on the GPU. Please refer to the [Supported Operators guide](https://nvidia.github.io/spark-rapids/docs/supported_ops.html) for more details on limitations on UDFs and unsupported operators.\\n\",\n", + " \"The Qualification tool generates a report of the “Exec” in the “SparkPlan” or “Executor Nodes” along with the estimated acceleration on the GPU. Please refer to the [Supported Operators guide](https://nvidia.github.io/cudf-spark/docs/supported_ops.html) for more details on limitations on UDFs and unsupported operators.\\n\",\n", " \"\\n\",\n", " \"1. App ID\\n\",\n", " \"2. SQL ID\\n\",\n", @@ -715,7 +715,7 @@ " }\n", " ]\n", " },\n", - " \"notebookName\": \"[RAPIDS Accelerator for Apache Spark] Qualification Tool Notebook Template\",\n", + " \"notebookName\": \"[NVIDIA cuDF plugin for Apache Spark] Qualification Tool Notebook Template\",\n", " \"widgets\": {\n", " \"Eventlog Path\": {\n", " \"currentValue\": \"/dbfs/user1/qualification_logs\",\n", diff --git a/tools/emr/README.md b/tools/emr/README.md index 896c6cc70..90e835742 100644 --- a/tools/emr/README.md +++ b/tools/emr/README.md @@ -1,6 +1,6 @@ # EMR Qualification/Profiling Quick Start Notebooks -The RAPIDS Accelerator for Apache Spark includes two key tools for understanding the benefits of +The NVIDIA cuDF plugin for Apache Spark provides two key tools for understanding the benefits of GPU acceleration as well as analyzing GPU Spark jobs. For customers on EMR, the quick start notebooks offer a simple interface for running the tools given a set of Spark event logs from CPU (qualification) or GPU (profiling) application runs. diff --git a/tools/emr/[RAPIDS Accelerator for Apache Spark] Profiling Tool Notebook Template.ipynb b/tools/emr/[NVIDIA cuDF plugin for Apache Spark] Profiling Tool Notebook Template.ipynb similarity index 99% rename from tools/emr/[RAPIDS Accelerator for Apache Spark] Profiling Tool Notebook Template.ipynb rename to tools/emr/[NVIDIA cuDF plugin for Apache Spark] Profiling Tool Notebook Template.ipynb index 012c7e91b..31c027cdc 100644 --- a/tools/emr/[RAPIDS Accelerator for Apache Spark] Profiling Tool Notebook Template.ipynb +++ b/tools/emr/[NVIDIA cuDF plugin for Apache Spark] Profiling Tool Notebook Template.ipynb @@ -15,7 +15,7 @@ } }, "source": [ - "## Profiling Tool for the RAPIDS Accelerator for Apache Spark\n", + "## Profiling Tool for the NVIDIA cuDF plugin for Apache Spark\n", "\n", "To run the profiling tool, enter the log path that represents the location of your Spark GPU event logs. Then, select \"Run all\" to execute the notebook. Once the notebook completes, various output tables will appear below. For more options on running the profiling tool, please refer to the [Profiling Tool User Guide](https://docs.nvidia.com/spark-rapids/user-guide/latest/profiling/quickstart.html#running-the-tool).\n", "\n", @@ -634,7 +634,7 @@ } ] }, - "notebookName": "[RAPIDS Accelerator for Apache Spark] Profiling Tool Notebook Template" + "notebookName": "[NVIDIA cuDF plugin for Apache Spark] Profiling Tool Notebook Template" }, "kernelspec": { "display_name": "Python 3", diff --git a/tools/emr/[RAPIDS Accelerator for Apache Spark] Qualification Tool Notebook Template.ipynb b/tools/emr/[NVIDIA cuDF plugin for Apache Spark] Qualification Tool Notebook Template.ipynb similarity index 98% rename from tools/emr/[RAPIDS Accelerator for Apache Spark] Qualification Tool Notebook Template.ipynb rename to tools/emr/[NVIDIA cuDF plugin for Apache Spark] Qualification Tool Notebook Template.ipynb index f66e14fbe..2c84d41bd 100644 --- a/tools/emr/[RAPIDS Accelerator for Apache Spark] Qualification Tool Notebook Template.ipynb +++ b/tools/emr/[NVIDIA cuDF plugin for Apache Spark] Qualification Tool Notebook Template.ipynb @@ -15,7 +15,7 @@ } }, "source": [ - "## Qualification Tool for the RAPIDS Accelerator for Apache Spark\n", + "## Qualification Tool for the NVIDIA cuDF plugin for Apache Spark\n", "\n", "To run the qualification tool, enter the log path that represents the location of your Spark CPU event logs. Then, select \"Run all\" to execute the notebook. Once the notebook completes, various output tables will appear below. For more options on running the qualification tool, please refer to the [Qualification Tool User Guide](https://docs.nvidia.com/spark-rapids/user-guide/latest/qualification/quickstart.html#running-the-tool).\n", "\n", @@ -599,7 +599,7 @@ "2. Stage ID\n", "3. Average Speedup Factor: The average estimated speed-up of all the operators in the given stage.\n", "4. Stage Task Duration: The amount of time spent in tasks of SQL DataFrame operations for the given stage.\n", - "5. Unsupported Task Duration: The sum of task durations for the unsupported operators. For more details, see [Supported Operators](https://nvidia.github.io/spark-rapids/docs/supported_ops.html).\n", + "5. Unsupported Task Duration: The sum of task durations for the unsupported operators. For more details, see [Supported Operators](https://nvidia.github.io/cudf-spark/docs/supported_ops.html).\n", "6. Stage Estimated: Indicates if the stage duration had to be estimated (True or False).\n" ] }, @@ -652,7 +652,7 @@ "source": [ "## Execs Output\n", "\n", - "The Qualification tool generates a report of the “Exec” in the “SparkPlan” or “Executor Nodes” along with the estimated acceleration on the GPU. Please refer to the [Supported Operators guide](https://nvidia.github.io/spark-rapids/docs/supported_ops.html) for more details on limitations on UDFs and unsupported operators.\n", + "The Qualification tool generates a report of the “Exec” in the “SparkPlan” or “Executor Nodes” along with the estimated acceleration on the GPU. Please refer to the [Supported Operators guide](https://nvidia.github.io/cudf-spark/docs/supported_ops.html) for more details on limitations on UDFs and unsupported operators.\n", "\n", "1. App ID\n", "2. SQL ID\n", @@ -770,7 +770,7 @@ } ] }, - "notebookName": "[RAPIDS Accelerator for Apache Spark] Qualification Tool Notebook Template" + "notebookName": "[NVIDIA cuDF plugin for Apache Spark] Qualification Tool Notebook Template" }, "kernelspec": { "display_name": "Python 3",