Selective classification (also known as classification with reject option) conservatively extends a classifier with a selection function to determine whether or not a prediction should be accepted (i.e., trusted, used, deployed). This is a highly relevant issue in socially sensitive tasks, such as credit scoring. State-of-the-art approaches rely on Deep Neural Networks (DNNs) that train at the same time both the classifier and the selection function. These approaches are model-specific and computationally expensive. We propose a model-agnostic approach, as it can work with any base probabilistic binary classification algorithm, and it can be scalable to large tabular datasets if the base classifier is so. The proposed algorithm, called SCROSS, exploits a cross-fitting strategy and theoretical results for quantile estimation to build the selection function. Experiments on real-world data show that SCROSS improves over existing methods.

A Model-Agnostic Heuristics for Selective Classification

Pugnana, Andrea
;
2023

Abstract

Selective classification (also known as classification with reject option) conservatively extends a classifier with a selection function to determine whether or not a prediction should be accepted (i.e., trusted, used, deployed). This is a highly relevant issue in socially sensitive tasks, such as credit scoring. State-of-the-art approaches rely on Deep Neural Networks (DNNs) that train at the same time both the classifier and the selection function. These approaches are model-specific and computationally expensive. We propose a model-agnostic approach, as it can work with any base probabilistic binary classification algorithm, and it can be scalable to large tabular datasets if the base classifier is so. The proposed algorithm, called SCROSS, exploits a cross-fitting strategy and theoretical results for quantile estimation to build the selection function. Experiments on real-world data show that SCROSS improves over existing methods.
2023
Settore INF/01 - Informatica
37th AAAI Conference on Artifical Intelligence
Washington DC, USA
February 7–14, 2023
PROCEEDINGS OF THE 37th AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE
AAAI Press
   Fairness and Intersectional Non-Discrimination in Human Recommendation
   FINDHR
   European Commission
   Horizon Europe Framework Programme
   101070212
File in questo prodotto:
File Dimensione Formato  
AModelAgnosticHeuristicsForSelectiveClassification.pdf

accesso aperto

Tipologia: Published version
Licenza: Solo Lettura
Dimensione 161.78 kB
Formato Adobe PDF
161.78 kB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11384/136443
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact