CG still doesn't have a good image quality measure - New measurements reveal weaknesses of image quality metrics in evaluating graphics artifacts

And near as I can tell, they aren't great for compression either. 

New measurements reveal weaknesses of image quality metrics in evaluating graphics artifacts

Martin Čadík, Robert Herzog, Rafał Mantiuk, Karol Myszkowski, Hans-Peter Seidel

Reliable detection of global illumination and rendering artifacts in the form of localized distortion maps is important for many graphics applications. Although many quality metrics have been developed for this task, they are often tuned for compression/transmission artifacts and have not been evaluated in the context of synthetic CG-images. In this work, we run two experiments where observers use a brush-painting interface to directly mark image regions with noticeable/objectionable distortions in the presence/absence of a high-quality reference image, respectively. The collected data shows a relatively high correlation between the with-reference and no-reference observer markings. Also, our demanding per-pixel image-quality datasets reveal weaknesses of both simple (PSNR, MSE, sCIE-Lab) and advanced (SSIM, MS-SSIM, HDR-VDP-2) quality metrics.