Amazon opens TorchNeuron beta so PyTorch code runs on Trainium chips with a one-line device change
The open-source backend supports Trainium2 and Trainium3, works with Hugging Face and vLLM plugins, and starts public beta on 20 October. Some PyTorch features are not supported yet.
Illustration · not a photoHow this story was checked
- ResearchedWith AI assistance · 2 sources
- AI source checkEach sentence matched to its source
- Editor approvalMayank Sahu · 20 Oct, 12:49
- Published20 Oct, 12:49
Verify this story
- 2 sources, 1 official · 2 with an archived copy · see sources
- Every fact has a label and a source number · fact ledger
- What we searched, read and could not verify · research note
- Machine-readable: facts as JSON · text as Markdown
How this was made: researched, drafted and source-checked by AI (Claude, made by Anthropic), then published under the editor’s standing approval. A person did not read it line by line before it went live. AI policy
In short
Amazon Web Services has released TorchNeuron, an open-source native PyTorch backend that lets existing PyTorch code run on its Trainium AI chips by setting the device to “neuron”. It supports Trainium2 and Trainium3 and entered public beta on 20 October 2026 under the Apache 2.0 licence.
What TorchNeuron is
TorchNeuron is a native PyTorch backend for AWS Trainium, announced on the PyTorch blog by AWS engineers on 19 October ahead of PyTorch Conference North America in San Jose.12 It is open source under Apache 2.0 and entered public beta on 20 October.12
For most code, the change is moving the model to the new device: `model.to("neuron")`.1 Code written against PyTorch’s `torch.accelerator` API runs unchanged, AWS says.1 It supports Trainium2 and Trainium3.12
How it works
- Eager mode for research and debugging, where operations run immediately and normal Python debugging works.1
- torch.compile for production, which hands the model graph to the Neuron compiler.1
- Distributed training with DDP, FSDP2, tensor and pipeline parallelism.1
- Custom kernels through the Neuron Kernel Interface, with open-source FlashAttention and mixture-of-experts kernels.1
Hugging Face Transformers and Diffusers support Trainium as an upstream device, and vLLM plugins serve language and diffusion models.1
What is not ready yet
AWS lists gaps in the first release, including sparse tensors, NestedTensor, arbitrary attention masks and dynamic input shapes.12 FP8 and TorchAO are in private beta.1 An experimental adaptive eager mode aims to bring eager speed closer to compiled graphs.1
Early users
AWS says Cartesia and Deepgram are early users; Deepgram reports about 60% more audio processed per dollar than its GPU baselines in its own tests.1 AWS also says about 100,000 Hugging Face models ran in its September tests.1 These are company-reported figures.
Why it matters
Software, not just chips, has kept many teams on Nvidia GPUs. A backend that runs standard PyTorch code with few changes lowers the cost of trying other hardware.
Fact ledger
How labels workQuestions readers ask
What is TorchNeuron?
An open-source PyTorch backend from AWS that lets PyTorch models run on Trainium chips using the device name “neuron”.
Which Trainium chips does TorchNeuron support?
Trainium2 and Trainium3.
Is TorchNeuron free?
The software is open source under Apache 2.0. Using Trainium chips on AWS is paid.