When software is compiled with optimizations for the hardware architecture it is going to be run on, significant increases in speed and processing efficency can be expected. Furthermore, when a Linux distribution is built from scratch, you have full control over the software installed and running on the system, resulting in dramatically-hardened security. For these reasons it is better to use a custom Linux distribution built for your exact needs rather than rely on a pre-rolled distribution. Although various package management systems offer convenience, there is no alternative or equivalent to building software packages by compiling them from source code.
An excellent resource for learning the process of building a custom Linux distro is Linux From Scratch (http://www.linuxfromscratch.org/). I highly recommend this project for anyone interested in learning how Linux works.