PyTorch 2.15 is out, and its biggest change is under the hood: a rewritten NCCL backend now powers every multi-GPU job on NVIDIA hardware by default. The release also brings faster 4-bit model decoding on Blackwell chips, FP8 support on Apple Silicon, and wider AMD and Intel GPU coverage. It packs 3,117 commits from 583 contributors since version 2.14.
What PyTorch 2.15 Changes at a Glance
PyTorch is the framework behind much of today’s AI research and many production models. Each 2.x release has pushed it toward working well on any chip, not just one vendor’s hardware. Version 2.15 continues that push with updates for NVIDIA, Apple, AMD and Intel.
| Area | What is new in 2.15 |
|---|---|
| Multi-GPU training | Rewritten NCCL backend becomes the default, with a flight recorder for hangs |
| NVIDIA Blackwell | Faster 4-bit (NVFP4) decoding through NVGEMM |
| Apple Silicon | FP8 support, a faster decode path and public stream APIs |
| AMD | FlyDSL kernels on gfx950, early MI450 support, ROCm 10.0 wheels |
| Intel GPUs | Simpler SYCL kernels and multi-GPU profiling |
| Python and CUDA | Python 3.15 support, CUDA 13.2 default wheel |
Rewritten NCCL Backend Becomes the Default
When a model trains across many GPUs, those chips constantly swap data through a communication layer. On NVIDIA hardware that layer is NCCL. In PyTorch 2.15, every job using the nccl backend now runs on the rewritten version, which was an opt-in preview in 2.14.
If something breaks, teams can switch back by setting TORCH_DIST_USE_NCCL2=0 or by using the nccl-legacy backend name. Communicators are now created up front with the process group. Clusters that use NCCL 2.29 or newer also get a faster path for shrinking or growing a job without a full restart.
A Flight Recorder for Stuck Training Runs
At cluster scale, one stalled GPU can freeze an entire run, and finding the cause often meant recreating the failure by hand. A new flight recorder writes a post-mortem to disk when a collective operation hangs. If a timeout hits on any rank, every rank dumps its record, so engineers can see what each GPU was doing at the moment of the freeze.
PyTorch 2.15 Speeds Up Decoding on Blackwell and Mac
For running large language models, 2.15 tunes the NVGEMM backend for 4-bit NVFP4 decoding on NVIDIA Blackwell GPUs. It removes a separate small step per layer and lets kernels overlap more, which means fewer wasted launches while the model generates text. The CUDA 13.4 wheels also add experimental support for NVIDIA Rubin GPUs.
Apple Silicon users get several gains too:
- FP8 support (float8_e4m3fn) and scaled matrix multiplication on Apple GPUs
- A new grouped matrix multiply that lifts mixture-of-experts decoding by 1.32 to 1.34 times on an M3 Ultra
- A single-row softmax over a 128,256-word vocabulary that runs 8 times faster
- A public torch.mps.Stream API modeled on the one in CUDA
If you run open models on a Mac, these changes should make local generation feel quicker over time. Our earlier coverage of local AI agents shows why on-device speed matters more each month.
Broader AMD and Intel GPU Support
PyTorch is also widening its list of friendly chips. AMD gains FlyDSL as another kernel language the compiler can use on gfx950, plus early support for the MI450 generation and new ROCm 10.0 wheels. Intel GPUs get a simpler way to write SYCL kernels, event sharing across processes, and profiling that spans several GPUs.
This fits a pattern we have covered before. Amazon recently showed how TorchNeuron on Trainium lets PyTorch apps jump to its own chips, and Google and Meta did the same with TorchTPU support. Every major chip maker now wants PyTorch to run well on its silicon.
What This Means for You
Most people never touch PyTorch directly, but the apps and models you use often depend on it. If you do build or train models, a few practical steps are worth taking before you upgrade.
- Check your Python version, because Python 3.10 is no longer supported and 2.15 needs 3.11 or newer.
- Test multi-GPU jobs on a small run first, and keep the nccl-legacy setting handy as a rollback.
- Look at the changed gradient behavior in FSDP2, which alters defaults when reduce_dtype is not set.
- Mac users can try the new FP8 and decode paths for local model speed-ups.
The maintainers also plan a Q&A webinar on November 5, 2026 at 11 AM PT for questions about the release.
Sources
Primary source: PyTorch 2.15 release announcement.
Conclusion
PyTorch 2.15 is a practical release rather than a flashy one. The new default NCCL backend, the hang-debugging tools and the faster low-precision decoding all aim at making large AI jobs more reliable and cheaper to run. For more stories like this, browse our AI news page.
