Google and Meta have introduced TorchTPU, an open source backend that lets PyTorch run natively on Google TPUs. Announced on October 20, 2026, it promises that many existing PyTorch models can move to Google’s AI chips with a change as small as swapping “cuda” for “tpu” in the code.
That matters because PyTorch is the most popular framework for building AI models, while TPUs grew up around Google’s own tools. TorchTPU is meant to close that gap and give developers a real alternative to GPUs.
What TorchTPU Is and Why It Exists
TPUs, or Tensor Processing Units, are chips Google designs specifically for the math behind neural networks. Google uses them to train and serve its own models, including Gemini, and rents them out through Google Cloud.
The problem has always been software. TPUs were tied to the TensorFlow and JAX ecosystems, and PyTorch users had to rely on PyTorch/XLA, an older bridge that Google introduced in 2020.
The trouble with the old approach
According to the TorchTPU team, PyTorch/XLA required intrusive code changes, suffered from slow compile times and did not support PyTorch’s normal step-by-step “eager” mode. It also lagged behind new distributed training features.
TorchTPU was started at Google in mid 2025, working with the PyTorch team at Meta. It was built from the ground up as a proper PyTorch backend instead of a translation layer.
How TorchTPU Brings PyTorch to TPUs
The project follows a few clear goals. Eager execution comes first, torch.compile is supported, and familiar PyTorch distributed tools work as expected.
- Eager and compiled modes, plus a new hybrid “DeferAndFuse” mode that can run 50% or more faster than strict eager.
- Support for roughly 450 core PyTorch operations, covering the major deep learning architectures.
- Custom TPU kernels written in Pallas, with experimental support for Helion.
- Distributed training with DDP, FSDP and DTensor.
- Tested integration with vLLM, SGLang, TorchTitan, Ray and TorchRec.
Google says TorchTPU already runs production inference and training for some of its largest Cloud customers. Model coverage spans LLMs such as Qwen, Llama, Gemma and Mistral, plus speech and vision models like Whisper and CLIP.
TorchTPU Performance by the Numbers
The launch post shares a range of early results. Here are some of the headline figures.
| Test | Reported result |
|---|---|
| Models tested continuously | About 5,000 Hugging Face models |
| Dynamic-shape correctness check | 834 of 839 LLMs correct (99%) |
| Qwen3 MoE 235B on 64 TPU v7 chips | 1,828 tokens/sec vs 1,937 for JAX |
| Inference cost (8K/1K benchmark) | 2.01 cents per million tokens |
| Agentic workload cost | 0.83 cents per million tokens |
The training result is notable because it puts PyTorch close to JAX on Google’s own hardware. The cost figures come from Google’s submissions to third-party InferenceX and AgentX benchmarks, so treat them as company-reported numbers.
Hugging Face Support Arrives on Day One
Hugging Face announced support the same day. Its Transformers library can now load a model on a TPU by passing device_map=”tpu”, and Diffusers treats TPU as a first-class device for image and video pipelines.
Hugging Face says its older Optimum TPU package is no longer needed for Transformers and Diffusers models. It also reports that 96.5% of the Transformers tests that run on TPU now pass, and that 19 of the 34 issues it filed with the TorchTPU project are already fixed.
For PyTorch developers, TPUs stop being a special project and start looking like just another device option.
What This Means for You
If you build or fine-tune models, TorchTPU gives you a new hardware choice without rewriting your code. That could help with GPU shortages and cloud costs, which are growing concerns as AI workloads scale.
- Try a small model on a Cloud TPU VM before moving bigger jobs.
- Expect to tune shapes and sharding for peak speed, since a simple device swap only gets a model running.
- Watch the project’s issue tracker, as it is still in active development.
This follows a similar move from Amazon, which recently launched its own PyTorch backend for Trainium chips. If you are exploring options, our list of AI developer tools and our machine learning tools comparison are good next stops. You can also follow more updates in our AI news section.
Sources
Primary source: the TorchTPU announcement on the PyTorch blog. Additional details from Hugging Face.
Conclusion
TorchTPU is Google and Meta’s answer to a long-standing problem: PyTorch users could not easily use Google’s TPUs. The new backend runs PyTorch natively, supports eager and compiled modes, and already handles production workloads for large Cloud customers. Early numbers put it close to JAX on training and competitive on inference costs. With Hugging Face support built in from day one, TPUs just became a far more realistic option for everyday PyTorch developers.
