If you use X.509 server authentication with 2,048-bit RSA keys, tcpcrypt offers about a 25x speed-up over SSL for equivalent security. (Actually slightly better, since tcpcrypt offers forward secrecy while, in the benchmark, SSL does not.) The key optimization is batch signing, where a single RSA signature can authenticate a bunch of connections at once. There are graphs showing this in the paper and talk slides.