I had always assumed that it was due to the long exposure times of early photographic techniques such as the Daguerreotype.

Specifically, since photographic skill was in short supply, and people with equipment had bad experiences with blurry faces when people couldn't maintain their smiles long enough for the exposure, that a trend emerged advising people not to smile.

Then, as this trend became firmly entrenched and everyone forgot the original reason, an idea was collectively conjured that it simply wasn't "fashionable" to smile in a photograph.