Tensor Is the Might: Building a GPU-Accelerated Library from Scratch

Tensor Is the Might

I built a complete, accelerated tensor library from scratch in C to demystify the math behind neural networks. Starting with flat arrays and metadata, I implemented elementwise operations before tackling the complexities of GPU acceleration. To keep things manageable, I chose Metal for Apple Silicon, leveraging its unified memory to write custom kernels and achieve performance that CPU-only frameworks simply cannot match.

"I tried many times to get the most out of CPU-only tensors, but even with BLAS, LAPACK, OpenMP – I could not reach the same magnitude of performance as GPU-accelerated frameworks."