D. J. Bernstein
Fast arithmetic

The art of FFT benchmarking

Frigo and Johnson announced the ``Fastest Fourier Transform in the West'' in 1997, claiming that it was ``typically faster than all other publicly available DFT software'' and ``competitive with vendor-optimized FFTs, as demonstrated by our extensive benchmarks.'' Similar claims remained on the FFTW web pages until at least October 1999.

Unfortunately, the Frigo-Johnson benchmarks are highly deceptive:

How many other high-performance libraries have been excluded from the FFTW benchmarks, or slowed down by the FFTW authors?

The FFTW benchmark methodology has several other problems:

Format of results

Benchmarks should make it easy to answer three questions: The FFTW benchmark results are presented as graphs that are much less useful than the above tables: