Duarte Silva, A. P.2016-12-192016-12-192013DUARTE SILVA, A.P. - Comparing High-Dimensional Classifiers: Abuse and dangers of overall accuracy. In Symposium of the International Federation of Classification Societies IFCS 2013, Tilburg, Netherlands, 14-17 July, 2013. In Program and Book of abstracts, Conference of the International Federation of Classification Societies IFCS-2013. Netherlands: Understanding Society, 2013. [s.issn]. p.172http://hdl.handle.net/10400.14/21089Statistical classification has a respected tradition in the support of medical diagnosis. Early applications relied on classical methodologies that assumed training samples with more patients than disease predictors and understood that simple performance measures, that do not take into account disease prevalence and the different costs of negative and positive predictions, have serious limitations. More recently, new classification methodologies have been applied to large genomic data bases where thousands of genes are measured on a few dozen patients. However, many of the studies that have evaluated these proposals employed only overall accuracy measures. This practice is potentially misleading, as it is known that changing prior probabilities and/or cost assumptions can strongly affect the relative standing of traditional classification rules. This presentation describes a study on the consequences of comparing high-dimensional classification rules by different performance measures. It will be argued that measures based on expected utilities or decision curves, that focus on the precision of risk estimates near the optimal threshold, should be preferred to overall accuracy. Furthermore, it will be shown that when samples proportions are not close to true disease probabilities corrected by misclassification costs, the use of overall accuracy can indeed lead to incorrect rankings of high-dimensional classifiers.engClassifier evaluationDecision curvesHigh dimensional classificationMisclassification costsconference object