Field noteOct 21, 20261 min read1 source

Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.

In one minute

Hugging Face and AWS published the first part of a tutorial on system-level profiling for TorchNeuron, the native PyTorch backend for AWS Trainium.

What changed: Adds a hands-on guide to profiling TorchNeuron on Trainium, tracing operations from PyTorch dispatch through runtime scheduling to NeuronCore execution across three implementations.

The walkthrough configures torch.profiler with ProfilerActivity.PrivateUse1, NeuronConfig, and a Neuron trace export callback, then inspects the resulting data in Neuron Explorer.

The example profiles three implementations of RMSNorm using the same inputs and profiling schedule. A naive version made from individual PyTorch operations recorded 2.355 ms of Neuron device time across six operator calls. torch.rms_norm lowered to one fused Neuron kernel and recorded 403.3 microseconds. A naive function compiled with torch.compile(backend="neuron") ran as one compiled region at 375.5 microseconds.

The post also describes trace artifacts: a compiled Neuron executable is stored in a NEFF file, while its hardware instruction trace is stored in a paired NTFF file. Neuron Explorer can display framework dispatch, runtime scheduling, and NeuronCore execution.

The source does not establish performance for complete models, other operators, or other Trainium configurations. Its figures are measurements from this RMSNorm example, not a general Trainium benchmark.

Evidence / sources

  1. 1huggingface.co/blog/amazon/profiling-torchneuron-part-1
All Field Notes

Next reads

Read next

Open the graph
  1. 01topicObservabilityExplore the observability topic hub.
  2. 02topicInferenceExplore the inference topic hub.
  3. 03topicComputeExplore the compute topic hub.
  4. 04archiveField NotesOpen the latest editorial analysis.
  5. 05source ledgerSource LedgerInspect the public source evidence graph.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract