Stephen Jones, a leading expert and distinguished NVIDIA CUDA architect, offers his guidance and insights with a deep dive into the complexities of mapping applications onto massively parallel machines. Going beyond the basics to explore the intricacies of GPU programming, he focuses on practical techniques such as parallel program design and specific details of GPU optimization for improving the efficiency and performance of your application.
As part of an ongoing series, this session builds on previous talks. While there’s no requirement to have seen earlier sessions, you can explore foundational topics like how GPU computing works, how CUDA programming works, and how to write a CUDA program.
Whether you’re new to CUDA or looking to enhance your GPU programming skills, this session offers both the theoretical knowledge and actionable strategies needed to excel in high-performance computing.
Follow along with a PDF of the session, which will equip you with advanced skills and insights to write highly efficient CUDA programs, helping you get the most out of your GPUs. You’ll dive into:
- GPU architecture: Key differences between CPU and GPU approaches, with a focus on the NVIDIA Hopper H100 GPU and its implications for parallel processing.
- Parallelism: Distinction and effective use of data and task parallelism in CUDA programming.
- CUDA execution model: Understanding how CUDA manages threads and blocks to maximize performance.
- Optimizing data parallelism: Strategies for running bulk data parallelism and mitigating wave quantization issues.
- Single-wave kernels: Benefits of mapping data to threads for better load balancing and efficiency.
- Task parallelism: Enhancing efficiency using CUDA streams and managing dependencies between streams.
- Pipeline parallelism: Optimizing complex algorithms like sorting with data splitting and dependency management.
- Cache optimization: Techniques for tiling execution in cache and running tasks in series to boost performance.
- Advanced CUDA techniques: Avoiding cache thrashing, task-based cache tiling, and minimizing inter-task dependencies.
Watch the advanced talk on How To Write A CUDA Program, explore more videos on NVIDIA On-Demand, and gain valuable skills and insights from industry experts by joining the NVIDIA Developer Program.
This content was partially crafted with the assistance of generative AI and LLMs. It underwent careful review and was edited by the NVIDIA Technical Blog team to ensure precision, accuracy, and quality.
