Getting Better by Being Wrong