More Updates To The CUDA Handbook

Rewrote two chapters and pushed some big refactors

Just about every corner of The CUDA Handbook has been revised on the website now - if you haven’t look at it yet, now’s as good a time as any to check it out.

This week has been busy. The Scan and Reduction chapters have been rewritten to reflect current reality: warp intrinsics have now been around for long enough that we start our coverage with warp-sized problems before leading into block- and grid-wide ones.

The Scan chapter needed a significant update: we now lead with the Algebra Of Scans framing and, for clarity, got rid of the awkward Blelloch (upsweep/downsweep) formulation and now start with the algorithm of Duane Merrill’s Ph.D. thesis, which was the best-in-class algorithm at the time of publication of the First Edition. Merrill has since rendered that algorithm obsolete with the bandwidth-limited decoupled lookback formulation, published with Michael Garland, now available via the CUB and Thrust utility libraries. With Claude’s help, I was able to write a sample implementation of the decoupled lookback and port the stream compaction sample to use that algorithm, as well.

The Software Architecture chapter has been updated for currency. It now mentions the debugger and profiler, and coverage of versioning, cloud vendors, and other topics has been updated. I may add some coverage of profiling in other parts of the book, though profiling tutorials tend to seem rhetorical as opposed to possessing the investigative quality that motivates most usage of profilers.

I added coverage of NVRTC with a sample in the Reduction chapter and sample code, which prompted a repository-wide refactor of the error handling macros. Having a macro that imposed the policy of naming a local variable status had outstayed its welcome (the NVRTC sample caused namespace collisions in collocated CUDART and driver API code), so we renamed to status_cudart and Error_cudart. The driver API and NVRTC variants are status_cuda/Error_cuda and status_nvrtc/ Error_nvrtc, respectively.

I also got rid of some dated utility APIs in chLib: the threading API has been replaced by std::thread, and the high-resolution timing API has been replaced by std::chrono::steady_clock.

In terms of coverage of the topics outlined in the Table of Contents, the biggest gap now in the text and code is coverage of multi-GPU programming. The state of the art has advanced a great deal since publication of the First Edition, when multiple GPUs typically were made available on PCI Express via switches and bridge chips. Since then, NVIDIA has invested heavily in both scale-up and scale-out technologies, developing NVLink for cache-coherent chip interconnects and acquiring Mellanox for their best-in-class Infiniband network cards. Multi-GPU programming deserves a rewrite to cover these hardware technologies and the corresponding APIs.