Phonon Anharmonicity at the Limits of Perturbation Theory