Maximum Differentiation (MAD) Competition

Zhou Wang and Eero P. Simoncelli

MAD_fixmse_iters

MAD competition is an efficient methodology for comparing computational models of a perceptually discriminable quantity.� Rather than pre-selecting test stimuli for subjective evaluation and comparison to the models, stimuli are synthesized so as to optimally distinguish the models.� Specifically, we first synthesize a pair of stimuli that maximize/minimize one model while holding the other fixed.� We then repeat this procedure, but with the roles of the two models reversed.� Subjective testing on pairs of such synthesized stimuli provides a strong indication of the relative strengths and weaknesses of the two models.� Careful study of the stimuli may, in turn, suggest potential ways to improve a model or to combine aspects of multiple models. We demonstrate the methodology using two examples: contrast perception and perceptual image quality assessment.

Demonstration:

MAD competition between two full-reference image quality assessment models: Mean squared error (MSE) and Structural Similarity Index (SSIM)

Synthesizing maximum/minimum SSIM images along the equal-MSE contour in the image space:

mad2.gif

 

Synthesizing maximum/minimum SSIM images at different MSE levels:

 

fixmse.gif

Reference:

Zhou Wang and Eero P. Simoncelli, "Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual discriminability," Journal of Vision, vol. 8, no. 12, pp. 1-13, Sept. 2008.

 

Created Nov. 21, 2007, Updated Sept. 25, 2008