dctmtx Return the DCT transformation matrix of size N-by-N.