Sure. Basically everything in https://github.com/tysam-code/hlb-CIFAR10 was directly founded on concepts shared in the above paper, down to the coding, commenting, and layout styles (hence why I advocate so strongly for it as a requirement for ML. The empirical benefits are clear to me).
Before I sat down and wrote my first line, I spent a very long time thinking about how to optimize the repo. Not just in terms of information flow during training, but how the code was laid out (minimize the expected value of deltas for changes from a superset of possible code changes), to even the explanatory comments (ratio of space vs mental effort to decode the repo for experienced vs inexperienced developers). I really want it to be a good exemplary model of a different, more scalable, and more efficient way of conducting small-scale (and potentially resource-constrained) research. To do that, you have to maximize information efficiency at every stage of the pipeline, including temporally (!!!!).
It's not perfect, but I've used info theory as a strong guiding light for that repo. There's more to say here, but it's a long conversation about the expected utility of doing research a few different kinds of ways.
Before I sat down and wrote my first line, I spent a very long time thinking about how to optimize the repo. Not just in terms of information flow during training, but how the code was laid out (minimize the expected value of deltas for changes from a superset of possible code changes), to even the explanatory comments (ratio of space vs mental effort to decode the repo for experienced vs inexperienced developers). I really want it to be a good exemplary model of a different, more scalable, and more efficient way of conducting small-scale (and potentially resource-constrained) research. To do that, you have to maximize information efficiency at every stage of the pipeline, including temporally (!!!!).
It's not perfect, but I've used info theory as a strong guiding light for that repo. There's more to say here, but it's a long conversation about the expected utility of doing research a few different kinds of ways.