Running RDMA (Remote Direct Memory Access) GPU Workloads on OKE

This guide provides instructions for deploying and managing GPU workloads with RDMA connectivity on Oracle Cloud Infrastructure Kubernetes Engine (OKE). OKE is a fully-managed, scalable, and highly available Kubernetes service that enables you to deploy containerized applications to the cloud.

Supported Operating Systems

• Ubuntu 22.04
• Ubuntu 24.04
• Oracle Linux 8 (except for the GPU with RDMA & GPU Memory Cluster worker pools)

Required Policies

The following policies are required. The OCI Resource Manager stack will create them for you if you have the necessary permissions. If you don't have the permissions, please refer to the policy documentation below.

• Policy Configuration for Cluster Creation and Deployment
• Creating a Dynamic Group and a Policy for Self-Managed Nodes

Deploying an OKE Cluster with GPUs and RDMA Connectivity

You will need a CPU pool and a GPU pool. The OCI Resource Manager stack deploys a system worker pool by default, and you can choose to deploy additional CPU/GPU worker pools.

Note

Use the images listed in Images to Use for Worker Nodes for all worker pools in the cluster (system, CPU, GPU, and RDMA). These images include GPU drivers, the Lustre client, and other components required by this stack.

Deploy the Cluster

You can easily deploy the cluster with the Deploy to Oracle Cloud button below, which uses OCI Resource Manager. If you prefer deploying with Terraform locally, you can use the templates in the terraform directory.

Access the Cluster

You can access the cluster locally by downloading the kubeconfig file.

Alternatively, the template deploys an operator instance with the kubeconfig pre-configured and tools like Helm and k9s pre-installed. You can find the SSH command to access the operator node under the Application information tab in the OCI Resource Manager stack.

You can also access the cluster directly from your local machine via the OCI Bastion Service. The template creates a bastion service and provides a ready-to-run command under the Application information tab. See Accessing a Private OKE Cluster via OCI Bastion Service for details.

Verify Node Availability

Wait until all nodes are ready in the cluster:

kubectl get nodes

NAME           STATUS     ROLES    AGE     VERSION
10.0.103.73    Ready      <none>   2d23h   v1.35.2
10.0.127.206   Ready      node     2d3h    v1.35.2
10.0.127.32    Ready      node     2d3h    v1.35.2
10.0.83.93     Ready      <none>   2d23h   v1.35.2
10.0.96.82     Ready      node     2d23h   v1.35.2

Using RDMA Network Interfaces in Manifests

To use the RDMA interfaces in your pods, see Using RDMA Network Interfaces in Manifests for the required manifest sections and complete examples using hostNetwork or SR-IOV virtual functions.

Optional: Deploy Kueue & MPI Operator to Run NCCL Tests

See Running NCCL and RCCL Tests with Kueue and MPI Operator for deployment steps, per-shape test manifests, and example output.

Documentation

See the documentation index for guides on health checks, running NCCL/RCCL tests, RDMA networking, the Slurm Operator, and more.

Contributing

This project welcomes contributions from the community. Before submitting a pull request, please review our contribution guide.

Security

Please consult the security guide for our responsible security vulnerability disclosure process.