Sam Stites

VAE interpretability

November 19, 2018

Here are some speculative thoughts to show my future self how naive I am at the end of 2018.

VAEs seem to have this ability to navigate between high scalability and better informed structure than conventional NNs. I have a feeling this means that it will be on the rise as “The Next Big Thing.” At least, this just means that the NN success saga is not coming to a close as far as stopping points. There are some effects of this: - I expect that more research papers will address VAEs. - I expect interpretability will be a more common metric in papers (whatever that takes the form of). - I expect that VAEs and transfer learning will be a thing that is comparable to current transfer learning. - I bet causal learning and logical programming will be on the rise, towards the end of this VAE surge.

I think the most interesting thing here will be that some VAE/classical-ML architecture which will be used to make complex graphical models competitive again with NNs. Hopefully, this will end up automating the feature engineering of graphical models (for interpretability) and achieving scale. I’d have to read more about the automated statistician to find out how extensive it is, but I am assuming that it is not too crazy. Maybe we’ll all end up using some “Neural-VAE non-parametric graphical architecture search” (so… NVAENPGAS?) which will be practical enough to solve all current academic benchmarks at low cost.