Wednesday, June 06, 2012

Spotted: Rethinking statistical analysis methods for CHI

Rethinking statistical analysis methods for CHI

Maurits Kaptein, Judy Robertson

CHI researchers typically use a significance testing approach to statistical analysis when testing hypotheses during usability evaluations. However, the appropriateness of this approach is under increasing criticism, with statisticians, economists, and psychologists arguing against the use of routine interpretation of results using "canned" p values. Three problems with current practice - the fallacy of the transposed conditional, a neglect of power, and the reluctance to interpret the size of effects - can lead us to build weak theories based on vaguely specified hypothesis, resulting in empirical studies which produce results that are of limited practical or scientific use.

No comments:

Post a Comment