Webtagr - Technology News Summarizer

Functions are Vectors: Extending Linear Algebra to Infinite Dimensions

2025-07-06

This article explores the concept of functions as infinite-dimensional vectors, demonstrating how the tools of linear algebra can be applied to a wide range of problems, from image and geometry processing to curve fitting, light transport, and machine learning. Starting with finite-dimensional vector spaces, it progresses to infinite dimensions, proving that functions form a vector space. The article then delves into linear operators, differentiation, the Laplacian operator, and the spectral theorem's application in function spaces, culminating in application examples such as Fourier series, image compression, and spherical harmonics.

Differentiable Programming: A Deep Dive from First Principles

2025-04-17

This article provides a comprehensive explanation of differentiable programming, starting from the definition of derivatives in calculus and progressing to concepts like gradients, directional derivatives, and Jacobians. It details three differentiation methods: numerical differentiation, symbolic differentiation, and automatic differentiation (forward and reverse modes), comparing their strengths and weaknesses. Finally, it demonstrates how reverse-mode automatic differentiation, combined with gradient descent, can solve real-world optimization problems using an image de-blurring example.

Monte Carlo Sampling Crash Course: Rejection Sampling and Change of Coordinates

2025-04-14

This article introduces two crucial sampling techniques in Monte Carlo methods: rejection sampling and change of coordinates. Rejection sampling samples a simpler region and filters samples based on an acceptance probability to achieve sampling of a complex region. The article provides a detailed derivation of the probability density function for rejection sampling and extends it to non-uniform distributions. Change of coordinates utilizes the Jacobian determinant to map samples from a simple region to a complex region, enabling efficient sampling. The article uses the unit disk as an example, demonstrating how to achieve uniform sampling using polar coordinate transformation. Both methods have their advantages and disadvantages; rejection sampling is simple and easy to understand but its efficiency depends on the acceptance probability; change of coordinates is efficient but requires finding suitable coordinate transformations.