I don't know what to think about their protein sequence example.
.35 accuracy is sure "better than chance" but is it good enough for a real purpose? (e.g. pharmaceutical target screening) Is this a task which in principle could be done to near-perfect accuracy (did the patient in the case study die?) vs one which could not be (did they like the movie a lot, moderately, somewhat, almost, ?) How well can you do with simple heuristics?
You can follow the traces to the literature and find the answers to those questions and you will find that people are sanguine about fundamental issues such as "does this track of development converge on an asymptote of 99.8% accuracy or an asymptote of 75% accuracy."
Folks who know how to take a predictor and hook it up to a Kelly better make better money working for banks or trading on their own account than Google pays.
.35 accuracy is sure "better than chance" but is it good enough for a real purpose? (e.g. pharmaceutical target screening) Is this a task which in principle could be done to near-perfect accuracy (did the patient in the case study die?) vs one which could not be (did they like the movie a lot, moderately, somewhat, almost, ?) How well can you do with simple heuristics?
You can follow the traces to the literature and find the answers to those questions and you will find that people are sanguine about fundamental issues such as "does this track of development converge on an asymptote of 99.8% accuracy or an asymptote of 75% accuracy."
Folks who know how to take a predictor and hook it up to a Kelly better make better money working for banks or trading on their own account than Google pays.