Highly Efficient Matrix Transpose in Mojo: Beating CUDA?
2025-06-06
This blog post details how to implement a highly efficient matrix transpose kernel on the Hopper architecture using Mojo. The author walks through optimizations, starting from a naive approach and culminating in a kernel achieving 2775.49 GB/s bandwidth—competitive with, and potentially exceeding, equivalent CUDA implementations. Optimizations include using TMA (Tensor Map Access) descriptors, shared memory optimizations, data swizzling, and thread coarsening. The post dives into the implementation details and performance gains of each technique, providing complete code examples.