Efficient Backpropagation: Simplifying Linear Transformation Derivatives with Einsum

This post presents a clever backpropagation trick that simplifies the derivation of any einsum expression through a simple letter swap. Einsum is a concise way to express linear transformations such as matrix multiplication, dot products, Hadamard products, and more. The article uses matrix multiplication as an example, showing how to perform forward and backward propagation using einsum and verifying the results with JAX. This method avoids complex derivations, significantly simplifying backpropagation calculations in deep learning.