| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 1.63 MB | Adobe PDF |
Authors
Advisor(s)
Abstract(s)
Statistical classification has a respected tradition in the support of medical diagnosis.
Early applications relied on classical methodologies that assumed training samples with
more patients than disease predictors and understood that simple performance measures,
that do not take into account disease prevalence and the different costs of negative and
positive predictions, have serious limitations.
More recently, new classification methodologies have been applied to large genomic
data bases where thousands of genes are measured on a few dozen patients. However,
many of the studies that have evaluated these proposals employed only overall accuracy
measures. This practice is potentially misleading, as it is known that changing prior
probabilities and/or cost assumptions can strongly affect the relative standing of traditional
classification rules.
This presentation describes a study on the consequences of comparing high-dimensional
classification rules by different performance measures. It will be argued that measures
based on expected utilities or decision curves, that focus on the precision of risk
estimates near the optimal threshold, should be preferred to overall accuracy. Furthermore,
it will be shown that when samples proportions are not close to true disease probabilities
corrected by misclassification costs, the use of overall accuracy can indeed lead
to incorrect rankings of high-dimensional classifiers.
Description
Keywords
Classifier evaluation Decision curves High dimensional classification Misclassification costs
Pedagogical Context
Citation
DUARTE SILVA, A.P. - Comparing High-Dimensional Classifiers: Abuse and dangers of overall accuracy. In Symposium of the International Federation of Classification Societies IFCS 2013, Tilburg, Netherlands, 14-17 July, 2013. In Program and Book of abstracts, Conference of the International Federation of Classification Societies IFCS-2013. Netherlands: Understanding Society, 2013. [s.issn]. p.172