Does this relate to the addition theorem of Fourier Transforms? The weights of his NN looked not only looked like sine waves but were crudely orthogonal.

Maybe in the future we will think of the word "digital" as archaic and "analog" will be the moniker for advanced tech.