I came accross this discussion on Automated Machine Learning that might be of interest. The idea here is that the algorithm choice becomes part of a fully automated pipeline (not only the hyperparameters). I like the fully data driven aspects of it inlight of our OHDSI objectives and the presentation given by @hripcsa at the symposium.
There is a long history of efforts like this in Statistics dating back at
least to the 1980’s.
I’m not sure what impact any of this had but it was much discussed back
when I was a graduate student.
Hand, D. J. (1984). Statistical expert systems: design. The Statistician,
351-369.
Hand, D. J. (1985). Statistical expert systems: necessary attributes.
Journal of Applied Statistics, 12(1), 19-27.
Nelder, J.A. (1991). GLIMPSE, a knowledge-based front end for GLIM. In: IMA
Volume in Mathematics and its Applications 36: Computing and Graphics in
Statistics (Ed. A. Buja & P.A. Tukey), pp. 125–131. New York: Springer
Verlag.
Yes i think the idea is definitely not new, but recently there seem to be some nice results with this looking at NIPS conference, some contests and the python libraries that have been developed.