Rename RAPIDS Accelerator for Apache Spark to NVIDIA cuDF plugin for Apache Spark

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,6 +1,6 @@
 ---
 name: Bug report
-about: Create a bug report to help us improve the RAPIDS Accelerator JNI for Apache Spark
+about: Create a bug report to help us improve the NVIDIA cuDF plugin JNI for Apache Spark
 title: "[BUG]"
 labels: "? - Needs Triage, bug"
 assignees: ''

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,14 +1,14 @@
 ---
 name: Feature request
-about: Suggest an idea for the RAPIDS Accelerator JNI for Apache Spark
+about: Suggest an idea for the NVIDIA cuDF plugin JNI for Apache Spark
 title: "[FEA]"
 labels: "? - Needs Triage, feature request"
 assignees: ''
 
 ---
 
 **Is your feature request related to a problem? Please describe.**
-A clear and concise description of what the problem is. Ex. I wish the RAPIDS Accelerator JNI for Apache Spark would [...]
+A clear and concise description of what the problem is. Ex. I wish the NVIDIA cuDF plugin JNI for Apache Spark would [...]
 
 **Describe the solution you'd like**
 A clear and concise description of what you want to happen.

.github/ISSUE_TEMPLATE/submit-question.md

@@ -1,6 +1,6 @@
 ---
 name: Submit question
-about: Ask a general question about RAPIDS Accelerator JNI for Apache Spark
+about: Ask a general question about NVIDIA cuDF plugin JNI for Apache Spark
 title: "[QST]"
 labels: "? - Needs Triage, question"
 assignees: ''

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,6 +1,6 @@
 <!--
 
-Thank you for contributing to RAPIDS Accelerator for Apache Spark!
+Thank you for contributing to NVIDIA cuDF plugin for Apache Spark!
 
 Here are some guidelines to help the review process go smoothly.
 

.github/workflows/sync-clang-format-version.yml

@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-# A workflow to sync the clang-format version between spark-rapids-jni and cudf
+# A workflow to sync the clang-format version between cudf-spark-jni and cudf
 name: sync clang-format version with cudf
 
 on:
@@ -60,8 +60,8 @@ jobs:
         env:
           GH_TOKEN: ${{ secrets.AUTOMERGE_TOKEN }}
           NEW_VER: ${{ steps.update_clang_format_version.outputs.new_ver }}
-          GIT_AUTHOR_NAME: "spark-rapids automation"
-          GIT_COMMITTER_NAME: "spark-rapids automation"
+          GIT_AUTHOR_NAME: "cudf-spark automation"
+          GIT_COMMITTER_NAME: "cudf-spark automation"
           GIT_AUTHOR_EMAIL: "70000568+nvauto@users.noreply.github.com"
           GIT_COMMITTER_EMAIL: "70000568+nvauto@users.noreply.github.com"
         run: |

.pre-commit-config.yaml

@@ -32,7 +32,7 @@ repos:
           cudf_ver=$(grep -A1 "repo: $clang_format_pattern" "thirdparty/cudf/.pre-commit-config.yaml" | grep "rev:" | tail -1 | sed "s/.*rev: *//");
           if [ -z "$our_ver" ] || [ -z "$cudf_ver" ] || [ "$our_ver" != "$cudf_ver" ]; then
             echo "ERROR: clang-format version mismatch or failed to extract version!";
-            echo "  spark-rapids-jni: ${our_ver:-(empty)}";
+            echo "  cudf-spark-jni: ${our_ver:-(empty)}";
             echo "  cudf:             ${cudf_ver:-(empty)}";
             echo "Please update .pre-commit-config.yaml to use the same rev as cudf";
             exit 1;

CODE_OF_CONDUCT.md

@@ -2,7 +2,7 @@
 
 ## Overview
 
-Define the code of conduct followed and enforced by the RAPIDS Accelerator for Apache Spark project
+Define the code of conduct followed and enforced by the NVIDIA cuDF plugin for Apache Spark project
 
 ### Intended audience
 
@@ -42,7 +42,7 @@ This Code of Conduct applies both within project spaces and in public spaces whe
 
 ## Enforcement
 
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at  [spark-rapids-conduct@nvidia.com](mailto:spark-rapids-conduct@nvidia.com)  All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at  [cudf-spark-conduct@nvidia.com](mailto:cudf-spark-conduct@nvidia.com)  All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
 
 Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project’s leadership.
 

NOTICE

@@ -1,5 +1,5 @@
-RAPIDS Accelerator JNI For Apache Spark
-Copyright (c) 2022-2024, NVIDIA CORPORATION
+NVIDIA cuDF plugin JNI for Apache Spark
+Copyright (c) 2022-2026, NVIDIA CORPORATION
 
 --------------------------------------------------------------------------------
 

README.md

@@ -1,9 +1,13 @@
-# RAPIDS Accelerator JNI For Apache Spark
+# NVIDIA cuDF plugin JNI for Apache Spark
 
 This repository contains native support code for the
-[RAPIDS Accelerator for Apache Spark](https://github.com/NVIDIA/spark-rapids).
+[NVIDIA cuDF plugin for Apache Spark](https://github.com/NVIDIA/cudf-spark).
 
-[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/spark-rapids-jni)
+[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/NVIDIA/cudf-spark-jni)
+
+Note: The NVIDIA cuDF plugin for Apache Spark was formerly known as the RAPIDS Accelerator for
+Apache Spark.  The RAPIDS name will be sunset over time.  GitHub links from `spark-rapids-jni` will
+redirect to `cudf-spark-jni`.  Artifact names will remain the same for now.
 
 ## Building From Source
 

build/Dockerfile.devel

@@ -15,7 +15,7 @@
 #
 
 ###
-# Build the image for spark-rapids-jni local development environment.
+# Build the image for cudf-spark-jni local development environment.
 
 # Inherit from the CICD docker image
 ARG JNI_DOCKER_CICD_IMAGE

build/apply-patches

@@ -32,7 +32,7 @@ PATCH_DIR=$(realpath "$PATCH_DIR")
 CUDF_DIR=${CUDF_DIR:-$(realpath "$BASE_DIR/thirdparty/cudf/")}
 
 # Apply pattches to CUDF is problematic in a number of ways. But ultimately it comes down to
-# making sure that a user can do development work in spark-rapids-jni without the patches
+# making sure that a user can do development work in cudf-spark-jni without the patches
 # getting in the way
 # The operations I really want to support no matter what state CUDF is in are
 # 1) Build the repo from scratch

build/build-in-docker

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 #
-# Copyright (c) 2022-2025, NVIDIA CORPORATION. All rights reserved.
+# Copyright (c) 2022-2026, NVIDIA CORPORATION. All rights reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -16,7 +16,7 @@
 # limitations under the License.
 #
 
-# Build the spark-rapids-jni artifact in a Docker container with devtoolset
+# Build the cudf-spark-jni artifact in a Docker container with devtoolset
 
 set -e
 

build/buildcpp.sh

@@ -16,7 +16,7 @@
 #
 
 #
-# Script to build native code in cudf and spark-rapids-jni
+# Script to build native code in cudf and cudf-spark-jni
 #
 
 set -e
@@ -163,7 +163,7 @@ cmake --build "$LIBCUDFJNI_BUILD_PATH" "-j$CPP_PARALLEL_LEVEL"
 #
 mkdir -p "$SPARK_JNI_BUILD_PATH"
 cd "$SPARK_JNI_BUILD_PATH"
-echo "Configuring spark-rapids-jni native libs"
+echo "Configuring cudf-spark-jni native libs"
 CUDF_ROOT="$CUDF_PATH" \
   CUDF_INSTALL_DIR="$LIBCUDF_INSTALL_PATH" \
   CUDFJNI_BUILD_DIR="$LIBCUDFJNI_BUILD_PATH" \
@@ -183,5 +183,5 @@ CUDF_ROOT="$CUDF_PATH" \
 
 create_compile_commands_symlink "$SPARK_JNI_BUILD_PATH" "$PROJECT_BASE_DIR/src/main/cpp"
 
-echo "Building spark-rapids-jni native libs"
+echo "Building cudf-spark-jni native libs"
 cmake --build "$SPARK_JNI_BUILD_PATH" "-j$CPP_PARALLEL_LEVEL"

build/unapply-patches

@@ -37,7 +37,7 @@ if [ -d "$PATCH_DIR" ] ; then
 fi
 
 # Apply pattches to CUDF is problematic in a number of ways. But ultimately it comes down to
-# making sure that a user can do development work in spark-rapids-jni without the patches
+# making sure that a user can do development work in cudf-spark-jni without the patches
 # getting in the way
 # The operations I really want to support no matter what state CUDF is in are
 # 1) Build the repo from scratch

ci/Dockerfile

@@ -15,7 +15,7 @@
 #
 
 ###
-# Build the image for spark-rapids-jni CICD environment.
+# Build the image for cudf-spark-jni CICD environment.
 #
 # Arguments: CUDA_VERSION=[12.X.Y], OS_RELEASE=[8, 9], TARGETPLATFORM=[linux/amd64, linux/arm64]
 #

ci/Jenkinsfile.premerge

@@ -17,7 +17,7 @@
 
 /**
  *
- * Jenkinsfile for building spark-rapids-jni on blossom
+ * Jenkinsfile for building cudf-spark-jni on blossom
  *
  */
 import hudson.model.Result

ci/nightly-build.sh

@@ -48,7 +48,7 @@ fi
 if [ "${CUDA_VER}" == "cuda13" ]; then
   BUILD_FAULTINJ="OFF"
   BUILD_PROFILER="OFF"
-  # Disable sanitizer https://github.com/NVIDIA/spark-rapids-jni/issues/4127
+  # Disable sanitizer https://github.com/NVIDIA/cudf-spark-jni/issues/4127
   USE_SANITIZER="OFF"
 fi
 

ci/submodule-sync.sh

@@ -28,8 +28,8 @@ REPO=${REPO:-"cudf-spark-jni"}
 PARALLEL_LEVEL=${PARALLEL_LEVEL:-4}
 REPO_LOC="github.com/${OWNER}/${REPO}.git"
 
-GIT_AUTHOR_NAME="spark-rapids automation"
-GIT_COMMITTER_NAME="spark-rapids automation"
+GIT_AUTHOR_NAME="cudf-spark automation"
+GIT_COMMITTER_NAME="cudf-spark automation"
 GIT_AUTHOR_EMAIL="70000568+nvauto@users.noreply.github.com"
 GIT_COMMITTER_EMAIL="70000568+nvauto@users.noreply.github.com"
 git submodule update --init --recursive

docs/memory_management.md

@@ -1,12 +1,12 @@
 ## Memory Management Overview
 
 Effective memory management is crucial for processing queries successfully with limited memory resources.
-The Spark-RAPIDS plugin leverages the [RAPIDS Memory Manager (RMM)](https://github.com/rapidsai/rmm) to handle and recover from out-of-memory (OOM) errors during query processing. This document describes the mechanisms and the state management implemented in [SparkResourceAdaptorJni.cpp](../src/main/cpp/src/SparkResourceAdaptorJni.cpp). The plugin tracks every memory allocation and deallocation request to handle various OOM situations. It chooses the appropriate recovery mechanism, such as spilling, rollback-and-retry, or split-and-retry, based on the situation. If recovery is not possible, the plugin fails gracefully.
+The NVIDIA cuDF plugin for Apache Spark leverages the [RAPIDS Memory Manager (RMM)](https://github.com/rapidsai/rmm) to handle and recover from out-of-memory (OOM) errors during query processing. This document describes the mechanisms and the state management implemented in [SparkResourceAdaptorJni.cpp](../src/main/cpp/src/SparkResourceAdaptorJni.cpp). The plugin tracks every memory allocation and deallocation request to handle various OOM situations. It chooses the appropriate recovery mechanism, such as spilling, rollback-and-retry, or split-and-retry, based on the situation. If recovery is not possible, the plugin fails gracefully.
 
 
 ### Handling Out-of-Memory Errors
 
-The Spark-RAPIDS plugin manages both device memory and host memory (optional). It tracks all memory allocations to detect OOM errors. While an allocation request succeeds, the plugin does not interfere with the running threads. However, when the allocation request fails due to insufficient memory, the plugin pauses the requesting thread and allows it to retry later when more memory becomes available. The plugin employs several strategies to free up memory:
+The cuDF plugin manages both device memory and host memory (optional). It tracks all memory allocations to detect OOM errors. While an allocation request succeeds, the plugin does not interfere with the running threads. However, when the allocation request fails due to insufficient memory, the plugin pauses the requesting thread and allows it to retry later when more memory becomes available. The plugin employs several strategies to free up memory:
 
 - Spilling: Data marked as spillable is moved out of memory.
 - Rollback: If no thread can make progress even after spilling, the plugin starts rolling back threads to the point where their inputs are spillable, allowing other thread to proceed.
@@ -16,7 +16,7 @@ If no further splitting is possible, the plugin gracefully cancels the query and
 
 ### State Machine for OOM Handler
 
-To handle various OOM situations, the Spark-RAPIDS plugin keeps track of the state of individual threads. Note that one Spark task can use multiple threads during execution.
+To handle various OOM situations, the cuDF plugin keeps track of the state of individual threads. Note that one Spark task can use multiple threads during execution.
 
 A thread can have one of these states at a time:
 
@@ -37,7 +37,7 @@ The thread state can change based on the diagram below. Note that the thread sta
 
 ### Thread Priority
 
-The Spark-RAPIDS plugin uses thread priority to break ties between threads.
+The cuDF plugin uses thread priority to break ties between threads.
 Note that the thread priority is currently decoupled from query priority. Each task thread is assigned a priority based on their `task_id` and `thread_id`. 
 Shuffle threads have the highest priority to avoid priority inversion as the task threads may depend on the shuffle indirectly.
 
@@ -58,7 +58,7 @@ The selected thread transitions its state to `THREAD_SPLIT_THROW` and throws an
 From the view of the OOM state machine, each task has one or more "dedicated threads", along with zero or more "pool threads" (background threads). When checking whether a task is blocked, OOM state machine is lenient on dedicated threads (only require any one of the dedicated threads to be blocked), but stringent on pool threads (all pool threads must be blocked). Being treated leniently is not always a good thing, it increases the chance of being mistakenly identified as a blocked task, thus causing unnecessary deadlock resolution. So we don't want a thread to be treated as a dedicated thread unless it is really necessary. There are two ways of avoiding a thread being treated as a dedicated thread:
 
 1. Avoid calling TaskContext.setTaskContext() in the current thread, this will prevent OOM state machine connecting the current thread to the task as a dedicated thread. 
-2. Proactively register thread itself as a pool thread instead of a dedicated thread. An example can be found [here](https://github.com/NVIDIA/spark-rapids/blob/c39f6a6004b0cf684ca526172e87b2bd4481eb3a/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOrcScan.scala#L2056) for registering threads. (Don't forget to unregister the thread when it is done.)
+2. Proactively register thread itself as a pool thread instead of a dedicated thread. An example can be found [here](https://github.com/NVIDIA/cudf-spark/blob/c39f6a6004b0cf684ca526172e87b2bd4481eb3a/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOrcScan.scala#L2056) for registering threads. (Don't forget to unregister the thread when it is done.)
 
 In most cases, we recommend the second approach, because typically the main task and the background thread will form a producer-consumer relationship. The main task thread, which plays as a consumer, will typically wait for the background thread to produce data. If we choose approach 1 then while consumer is waiting, its Java thread state will be `WAITING`, and even if the producer is actively working (so the whole task should NOT be considered as blocked), the OOM state machine will mistakenly consider it as a blocked task because it cannot find any "pool thread" connected with this task. So "has at least one dedicated thread blocked on memory allocation, and all of the pool threads working on that task are also blocked" stands. 
 

pom.xml

@@ -23,11 +23,11 @@
   <artifactId>spark-rapids-jni</artifactId>
   <version>26.10.0-SNAPSHOT</version>
   <packaging>jar</packaging>
-  <name>RAPIDS Accelerator JNI for Apache Spark</name>
+  <name>NVIDIA cuDF plugin JNI for Apache Spark</name>
   <description>
-    Native code and CUDA kernels specific to the RAPIDS Accelerator for Apache Spark
+    Native code and CUDA kernels specific to the NVIDIA cuDF plugin for Apache Spark
   </description>
-  <url>http://github.com/NVIDIA/spark-rapids-jni</url>
+  <url>http://github.com/NVIDIA/cudf-spark-jni</url>
 
   <licenses>
     <license>
@@ -38,10 +38,10 @@
     </license>
   </licenses>
   <scm>
-    <connection>scm:git:https://github.com/NVIDIA/spark-rapids-jni.git</connection>
-    <developerConnection>scm:git:git@github.com:NVIDIA/spark-rapids-jni.git</developerConnection>
+    <connection>scm:git:https://github.com/NVIDIA/cudf-spark-jni.git</connection>
+    <developerConnection>scm:git:git@github.com:NVIDIA/cudf-spark-jni.git</developerConnection>
     <tag>HEAD</tag>
-    <url>https://github.com/NVIDIA/spark-rapids-jni</url>
+    <url>https://github.com/NVIDIA/cudf-spark-jni</url>
   </scm>
   <developers>
     <developer>

src/main/cpp/CMakeLists.txt

@@ -50,7 +50,7 @@ project(
 
 # We use no C++20 modules; disable module scanning so CMake emits no ninja dyndep phase. Otherwise
 # its dyndep refresh trips ninja's RefreshDyndepDependents assertion on incremental rebuilds after
-# a structural source change. Mirrors libcudf (rapidsai/cudf#19192).
+# a structural source change. Mirrors libcudf (nvidia/cudf#19192).
 set(CMAKE_CXX_SCAN_FOR_MODULES OFF)
 
 # ##################################################################################################
@@ -155,10 +155,10 @@ else()
 endif()
 
 # cuCollections
-# As of rapidsai/cudf PR #22277 (commit 5a375a3417ef70c55564bd0c4c6255a2da5dca49), libcudf
+# As of nvidia/cudf PR #22277 (commit 5a375a3417ef70c55564bd0c4c6255a2da5dca49), libcudf
 # wraps its `cuco::cuco` link with `$<BUILD_LOCAL_INTERFACE:...>`, so the cuco headers are no
 # longer transitively exposed to downstream consumers via `find_package(cudf)`. Several
-# spark-rapids-jni sources (e.g. the hash kernels) include cudf detail headers that pull in
+# cudf-spark-jni sources (e.g. the hash kernels) include cudf detail headers that pull in
 # `<cuco/hash_functions.cuh>`, so we need to fetch cuco ourselves.
 include(${rapids-cmake-dir}/cpm/cuco.cmake)
 rapids_cpm_cuco()

src/main/cpp/faultinj/README.md

@@ -22,7 +22,7 @@ The tool is a dynamically linked library `libcufaultinj.so` that is loaded by
 the CUDA process via CUDA Driver API `cuInit` if it is provided
 via the `CUDA_INJECTION64_PATH` environment variable.
 
-As an example it can be used to test RAPIDS Accelerator for Apache Spark.
+As an example it can be used to test the NVIDIA cuDF plugin for Apache Spark.
 Consult documentation to find how to set these variables correctly in the
 context of the framework under test.
 

src/main/cpp/profiler/README.md

@@ -1,6 +1,7 @@
-# Spark Rapids Profile Converter Tool
+# NVIDIA cuDF plugin for Apache Spark Profile Converter Tool
 
-This directory contains the Spark Rapids Profile Converter, a tool for converting NVTX profiling data from the Spark Rapids JNI library.
+This directory contains the NVIDIA cuDF plugin for Apache Spark Profile Converter, a tool for
+converting NVTX profiling data from the cuDF plugin JNI library.
 
 ## Building the Tool
 

src/main/cpp/profiler/spark_rapids_profile_converter.cpp

@@ -14,7 +14,7 @@
  * limitations under the License.
  */
 
-/* A tool that converts a spark-rapids profile binary into other forms. */
+/* A tool that converts a cuDF plugin profile binary into other forms. */
 
 #if 0
 #include <stdexcept>
@@ -114,7 +114,7 @@ void print_usage()
 {
   std::cout << "spark_rapids_profile_converter [OPTION]... profilebin" << std::endl;
   std::cout << R"(
-Converts the spark-rapids profile in profile.bin into other forms.
+Converts the cuDF plugin profile in profile.bin into other forms.
 
   -h, --help                show this usage message
   -j, --json                convert to JSON, default output is stdout
@@ -365,9 +365,7 @@ void verify_profile_header(std::ifstream& in)
   auto fb_ptr = read_flatbuffer(in);
   auto header = validate_fb<spark_rapids_jni::profiler::ProfileHeader>(*fb_ptr, "profile header");
   auto magic  = header->magic();
-  if (magic == nullptr) {
-    throw std::runtime_error("does not appear to be a spark-rapids profile");
-  }
+  if (magic == nullptr) { throw std::runtime_error("does not appear to be a cuDF plugin profile"); }
   if (magic->str() != "spark-rapids profile") {
     std::ostringstream oss;
     oss << "bad profile magic, expected 'spark-rapids profile' found '" << magic->str() << "'";