CUDA API Changes from 12.x to 13.x

This topic summarizes the CUDA API changes from version 12.x to 13.x, including Driver API and CUDART API signature changes, deprecated APIs, and removed header files.

Migration Details

Backward Compatible	No
Migration Path	Orin to Thor
Platform	NSR
OS	Linux and QNX
Migration Path	7.0 to 7.2

Driver API Signature Changes

The following table summarizes the Driver API signature changes:

Driver API Changes

API Function	12.x Signature	13.x Signature
cuMemPrefetchAsync	CUresult cuMemPrefetchAsync(CUdeviceptr devPtr, size_t count, CUdevice dstDevice, CUstream hStream)	CUresult cuMemPrefetchAsync(CUdeviceptr devPtr, size_t count, CUmemLocation location, unsigned int flags, CUstream hStream)
cuMemAdvise	CUresult cuMemAdvise(CUdeviceptr devPtr, size_t count, CUmem_advise advice, CUdevice device)	CUresult cuMemAdvise(CUdeviceptr devPtr, size_t count, CUmem_advise advice, CUmemLocation location)
cuDeviceGetUuid	Returns physical UUID	Returns physical UUID in non-MIG mode and MIG UUID in MIG mode
cuCtxCreate	CUresult cuCtxCreate(CUcontext *pctx, unsigned int flags, CUdevice dev)	CUresult cuCtxCreate(CUcontext *pctx, CUctxCreateParams *ctxCreateParams, unsigned int flags, CUdevice dev)
cuEventElapsedTime	Standard error checking	Relaxed error checking for performance improvement
cuStreamGetCaptureInfo	CUresult cuStreamGetCaptureInfo(CUstream hStream, CUstreamCaptureStatus *captureStatus_out, cuuint64_t *id_out, CUgraph *graph_out, const CUgraphNode **dependencies_out, size_t *numDependencies_out)	CUresult cuStreamGetCaptureInfo(CUstream hStream, CUstreamCaptureStatus *captureStatus_out, cuuint64_t *id_out, CUgraph *graph_out, const CUgraphNode **dependencies_out, const CUgraphEdgeData **edgeData_out, size_t *numDependencies_out)
cuStreamUpdateCaptureDependencies	CUresult cuStreamUpdateCaptureDependencies(CUstream hStream, CUgraphNode *dependencies, size_t numDependencies, unsigned int flags)	CUresult cuStreamUpdateCaptureDependencies(CUstream hStream, CUgraphNode *dependencies, const CUgraphEdgeData *dependencyData, size_t numDependencies, unsigned int flags)
cuGraphGetEdges	CUresult cuGraphGetEdges(CUgraph hGraph, CUgraphNode *from, CUgraphNode *to, size_t *numEdges)	CUresult cuGraphGetEdges(CUgraph hGraph, CUgraphNode *from, CUgraphNode *to, CUgraphEdgeData *edgeData, size_t *numEdges)
cuGraphNodeGetDependencies	CUresult cuGraphNodeGetDependencies(CUgraphNode hNode, CUgraphNode *dependencies, size_t *numDependencies)	CUresult cuGraphNodeGetDependencies(CUgraphNode hNode, CUgraphNode *dependencies, CUgraphEdgeData *edgeData, size_t *numDependencies)
cuGraphNodeGetDependentNodes	CUresult cuGraphNodeGetDependentNodes(CUgraphNode hNode, CUgraphNode *dependentNodes, size_t *numDependentNodes)	CUresult cuGraphNodeGetDependentNodes(CUgraphNode hNode, CUgraphNode *dependentNodes, CUgraphEdgeData *edgeData, size_t *numDependentNodes)
cuGraphAddDependencies	CUresult cuGraphAddDependencies(CUgraph hGraph, const CUgraphNode *from, const CUgraphNode *to, size_t numDependencies)	CUresult cuGraphAddDependencies(CUgraph hGraph, const CUgraphNode *from, const CUgraphNode *to, const CUgraphEdgeData *edgeData, size_t numDependencies)
cuGraphRemoveDependencies	CUresult cuGraphRemoveDependencies(CUgraph hGraph, const CUgraphNode *from, const CUgraphNode *to, size_t numDependencies)	CUresult cuGraphRemoveDependencies(CUgraph hGraph, const CUgraphNode *from, const CUgraphNode *to, const CUgraphEdgeData *edgeData, size_t numDependencies)
cuGraphAddNode	CUresult cuGraphAddNode(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, size_t numDependencies, CUgraphNodeParams *nodeParams)	CUresult cuGraphAddNode(CUgraphNode *phGraphNode, CUgraph hGraph, const CUgraphNode *dependencies, const CUgraphEdgeData *dependencyData, size_t numDependencies, CUgraphNodeParams *nodeParams)
cuMemcpyBatchAsync	CUresult cuMemcpyBatchAsync(CUdeviceptr *dsts, CUdeviceptr *srcs, size_t *sizes, size_t count, CUmemcpyAttributes *attrs, size_t *attrsIdxs, size_t numAttrs, size_t *failIdx, CUstream hStream)	CUresult cuMemcpyBatchAsync(CUdeviceptr *dsts, CUdeviceptr *srcs, size_t *sizes, size_t count, CUmemcpyAttributes *attrs, size_t *attrsIdxs, size_t numAttrs, CUstream hStream)
cuMemcpy3DBatchAsync	CUresult cuMemcpy3DBatchAsync(size_t numOps, CUDA_MEMCPY3D_BATCH_OP *opList, size_t *failIdx, unsigned long long flags, CUstream hStream)	CUresult cuMemcpy3DBatchAsync(size_t numOps, CUDA_MEMCPY3D_BATCH_OP *opList, unsigned long long flags, CUstream hStream)

CUDART API Signature Changes

The following table summarizes the CUDART API signature changes:

CUDART API Changes

API Function	12.x Signature	13.x Signature
cudaMemPrefetchAsync	cudaError_t cudaMemPrefetchAsync(const void *devPtr, size_t count, int dstDevice, cudaStream_t stream)	cudaError_t cudaMemPrefetchAsync(const void *devPtr, size_t count, struct cudaMemLocation location, unsigned int flags, cudaStream_t stream)
cudaMemAdvise	cudaError_t cudaMemAdvise(const void *devPtr, size_t count, enum cudaMemoryAdvise advice, int device)	cudaError_t cudaMemAdvise(const void *devPtr, size_t count, enum cudaMemoryAdvise advice, struct cudaMemLocation location)
cudaStreamGetCaptureInfo	cudaError_t cudaStreamGetCaptureInfo(cudaStream_t stream, enum cudaStreamCaptureStatus *captureStatus_out, unsigned long long *id_out)	cudaError_t cudaStreamGetCaptureInfo(cudaStream_t stream, enum cudaStreamCaptureStatus *captureStatus_out, unsigned long long *id_out, cudaGraph_t *graph_out, const cudaGraphNode_t **dependencies_out, const cudaGraphEdgeData **edgeData_out, size_t *numDependencies_out)
cudaStreamUpdateCaptureDependencies	cudaError_t cudaStreamUpdateCaptureDependencies(cudaStream_t stream, cudaGraphNode_t *dependencies, size_t numDependencies, unsigned int flags)	cudaError_t cudaStreamUpdateCaptureDependencies(cudaStream_t stream, cudaGraphNode_t *dependencies, const cudaGraphEdgeData *dependencyData, size_t numDependencies, unsigned int flags)
cudaGraphGetEdges	cudaError_t cudaGraphGetEdges(cudaGraph_t graph, cudaGraphNode_t *from, cudaGraphNode_t *to, size_t *numEdges)	cudaError_t cudaGraphGetEdges(cudaGraph_t graph, cudaGraphNode_t *from, cudaGraphNode_t *to, cudaGraphEdgeData *edgeData, size_t *numEdges)
cudaGraphNodeGetDependencies	cudaError_t cudaGraphNodeGetDependencies(cudaGraphNode_t node, cudaGraphNode_t *pDependencies, size_t *pNumDependencies)	cudaError_t cudaGraphNodeGetDependencies(cudaGraphNode_t node, cudaGraphNode_t *pDependencies, cudaGraphEdgeData *edgeData, size_t *pNumDependencies)
cudaGraphNodeGetDependentNodes	cudaError_t cudaGraphNodeGetDependentNodes(cudaGraphNode_t node, cudaGraphNode_t *pDependentNodes, size_t *pNumDependentNodes)	cudaError_t cudaGraphNodeGetDependentNodes(cudaGraphNode_t node, cudaGraphNode_t *pDependentNodes, cudaGraphEdgeData *edgeData, size_t *pNumDependentNodes)
cudaGraphAddDependencies	cudaError_t cudaGraphAddDependencies(cudaGraph_t graph, const cudaGraphNode_t *from, const cudaGraphNode_t *to, size_t numDependencies)	cudaError_t cudaGraphAddDependencies(cudaGraph_t graph, const cudaGraphNode_t *from, const cudaGraphNode_t *to, const cudaGraphEdgeData *edgeData, size_t numDependencies)
cudaGraphRemoveDependencies	cudaError_t cudaGraphRemoveDependencies(cudaGraph_t graph, const cudaGraphNode_t *from, const cudaGraphNode_t *to, size_t numDependencies)	cudaError_t cudaGraphRemoveDependencies(cudaGraph_t graph, const cudaGraphNode_t *from, const cudaGraphNode_t *to, const cudaGraphEdgeData *edgeData, size_t numDependencies)
cudaGraphAddNode	cudaError_t cudaGraphAddNode(cudaGraphNode_t *pGraphNode, cudaGraph_t graph, const cudaGraphNode_t *pDependencies, size_t numDependencies, struct cudaGraphNodeParams *nodeParams)	cudaError_t cudaGraphAddNode(cudaGraphNode_t *pGraphNode, cudaGraph_t graph, const cudaGraphNode_t *pDependencies, const cudaGraphEdgeData *dependencyData, size_t numDependencies, struct cudaGraphNodeParams *nodeParams)
cudaEventElapsedTime	Standard error checking	Relaxed error checking for performance improvement
cudaSignalExternalSemaphoresAsync	cudaError_t cudaSignalExternalSemaphoresAsync(const cudaExternalSemaphore_t *extSemArray, const struct cudaExternalSemaphoreSignalParams_v1 *paramsArray, unsigned int numExtSems, cudaStream_t stream)	cudaError_t cudaSignalExternalSemaphoresAsync(const cudaExternalSemaphore_t *extSemArray, const struct cudaExternalSemaphoreSignalParams *paramsArray, unsigned int numExtSems, cudaStream_t stream)
cudaWaitExternalSemaphoresAsync	cudaError_t cudaWaitExternalSemaphoresAsync(const cudaExternalSemaphore_t *extSemArray, const struct cudaExternalSemaphoreWaitParams_v1 *paramsArray, unsigned int numExtSems, cudaStream_t stream)	cudaError_t cudaWaitExternalSemaphoresAsync_v2(const cudaExternalSemaphore_t *extSemArray, const struct cudaExternalSemaphoreWaitParams *paramsArray, unsigned int numExtSems, cudaStream_t stream)
cudaMemcpyBatchAsync	cudaError_t cudaMemcpyBatchAsync(void **dptrs, void **srcs, size_t *sizes, size_t count, cudaMemcpyAttributes *attrs, size_t *attrIdxs, size_t numAttrs, size_t *failIdx, cudaStream_t stream)	cudaError_t cudaMemcpyBatchAsync(void **dptrs, void **srcs, size_t *sizes, size_t count, cudaMemcpyAttributes *attrs, size_t *attrIdxs, size_t numAttrs, cudaStream_t stream)
cudaMemcpy3DBatchAsync	cudaError_t cudaMemcpyBatch3DAsync(size_t numOps, cudaMemcpy3DBatchOp *opList, size_t failIdx, unsigned long long flags, CUstream stream)	cudaError_t cudaMemcpyBatch3DAsync(size_t numOps, cudaMemcpy3DBatchOp *opList, unsigned long long flags, CUstream stream)

Change Reasons and Examples

Key Memory Management API Changes

cuMemPrefetchAsync / cudaMemPrefetchAsync

• Reason: Add a CPU NUMA node parameter to CUDA managed memory APIs

• Before: Used CUdevice / int for device specification

• After: Uses CUmemLocation / struct cudaMemLocation for more flexible memory location specification

• Migration: Replace device parameter with location structure

cuMemAdvise / cudaMemAdvise

• Reason: Add a CPU NUMA node parameter to CUDA managed memory APIs

• Before: Used CUdevice / int for device specification

• After: Uses CUmemLocation / struct cudaMemLocation for location specification

• Migration: Replace device parameter with location structure

Graph API Enhancements

Multiple graph-related APIs now include CUgraphEdgeData / cudaGraphEdgeData parameters to support enhanced graph functionality:

• Affected APIs: All cuGraph* and cudaGraph* functions related to dependencies and edges

• Reason: Enhanced graph edge data support

• Migration: Add NULL / nullptr for edgeData parameter if not using edge data features

Performance Improvements

cuEventElapsedTime / cudaEventElapsedTime

• Reason: Performance improvement by relaxed error checking

• Change: Removed explicit context error checking for better performance

• Migration: No code changes required

Batched Memory Copy Changes

cuMemcpyBatchAsync / cudaMemcpyBatchAsync

• Reason: Batched Memcpy v2 improvements

• Change: Removed failIdx parameter

• Migration: Remove failIdx parameter from function calls

cuMemcpy3DBatchAsync / cudaMemcpy3DBatchAsync

• Reason: Batched Memcpy v2 improvements

• Change: Removed failIdx parameter

• Migration: Remove failIdx parameter from function calls

External Semaphore Updates

cudaSignalExternalSemaphoresAsync / cudaWaitExternalSemaphoresAsync

• Reason: Deprecate existing external semaphore structs and switch to structs binary compatible with driver structs

• Change: Updated parameter struct types

• Migration: Use new struct types for external semaphore operations

Deprecated APIs

Deprecated CUDART APIs

API Function	Status
CooperativeGroupLaunchMultiDevice	Deprecated in 13.x

Removed Header Files

The following unused header files have been removed in CUDA 13.x:

Removed Header Files

Header File	Replacement/Action
cuda_surface_types.h	Remove include statements
cuda_texture_types.h	Remove include statements
surface_functions.h	Remove include statements
texture_fetch_functions.h	Remove include statements

Migration Guidelines

1. Memory Management APIs: Update cuMemPrefetchAsync, cuMemAdvise, cudaMemPrefetchAsync, and cudaMemAdvise calls to use location structures instead of device IDs.

2. Graph APIs: Add NULL / nullptr parameters for new edgeData parameters if not using enhanced graph features.

3. Batch Copy APIs: Remove failIdx parameters from cuMemcpyBatchAsync, cuMemcpy3DBatchAsync, cudaMemcpyBatchAsync, and cudaMemcpy3DBatchAsync calls.

4. External Semaphores: Update struct types for external semaphore operations to use new binary-compatible versions.

5. Header Files: Remove includes for deprecated header files and update build scripts accordingly.

6. Context Creation: Update cuCtxCreate calls to use new CUctxCreateParams parameter.