Bad Optimizations Make Good Learning