Truncated SVD compresses a 1024×1024 moon image to 16× smaller with just 32 components
Truncated SVD is a matrix factorization technique that zeroes out all but the top n singular values, effectively reducing dimensionality. Applied to a 1024×1024 grayscale moon image, keeping only 32 of 1024 components yields a 16× reduction in storage while still resembling the moon. The error drops rapidly, showing that the first components capture most of the information, similar to Fourier-based compression. The post explores easy and hard cases and practical uses like data compression and low-rank approximation.
What’s remarkable here (the magic of PCA/SVD) is how quickly the error graph decreases. This proves that the first components capture most of the information present in M, while the trailing components only carry high-frequency/low-amplitude fine details.
- arjunchint
Nice write-up! The randomized SVD (Halko–Martinsson–Tropp) is a great practical fallback when you only need the top singular values/vectors.
- bee_rider
If you want a truncated SVD, there’s a decent chance you will be happy with a randomized SVD.
The paper “Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions” (Halko, Martinsson, and Tropp) is on arxiv. It is generally pretty accessible (although a little mathy), but also has great pseudocode for a number of common use-cases.
- nomel
It seems that decreases error curve always has the same shape. Is there a name for this shape? I assume you could sample just a few points and have a good estimate of what the error will be for any truncation.
And, this makes me naively assume you could apply some reversible transformation to the image, before applying this, to reduce some of the "perceptual" error (even if at the expense of numeric error)?