For training networks, designing en efficient chip is a scary proposition.
However for inference tasks, low precision GEMM (as you said) goes a long way and better than what you often get. That's why chips like Movidius' Myriad are getting popular and are more similar to DSPs than neuromorphic designs.
I agree that Intel's neuromorphic group doesn't get it, but other groups have taken neuromophic design principles that lead to efficient designs. For example, TrueNorth is very low precision, has great data locality, and though it was designed over 5 years ago can still use modern convolutional networks only imagined afterwards [0]. But its silicon implementation is not very brain like.
However for inference tasks, low precision GEMM (as you said) goes a long way and better than what you often get. That's why chips like Movidius' Myriad are getting popular and are more similar to DSPs than neuromorphic designs.
I agree that Intel's neuromorphic group doesn't get it, but other groups have taken neuromophic design principles that lead to efficient designs. For example, TrueNorth is very low precision, has great data locality, and though it was designed over 5 years ago can still use modern convolutional networks only imagined afterwards [0]. But its silicon implementation is not very brain like.
[0] https://arxiv.org/pdf/1603.08270.pdf