Chained deep learning using generalized cross-entropy for multiple annotators classification
Supervised learning requires the accurate labeling of instances, usually provided by an expert. Crowdsourcing platforms offer a practical and cost-effective alternative for large datasets when individual annotation is impractical. In addition, these platforms gather labels from multiple labelers. St...
- Autores:
-
Triana-Martinez, Jenniffer Carolina
Gil-González, Julian
Fernandez-Gallego, Jose A.
Lugo González, Carlos Andrés
- Tipo de recurso:
- Article of journal
- Fecha de publicación:
- 2023
- Institución:
- Universidad de Ibagué
- Repositorio:
- Repositorio Universidad de Ibagué
- Idioma:
- eng
- OAI Identifier:
- oai:repositorio.unibague.edu.co:20.500.12313/3837
- Acceso en línea:
- https://hdl.handle.net/20.500.12313/3837
- Palabra clave:
- Chained approach
Classification; deep learning
Generalized cross-entropy
Multiple annotators
- Rights
- openAccess
- License
- http://purl.org/coar/access_right/c_abf2
| Summary: | Supervised learning requires the accurate labeling of instances, usually provided by an expert. Crowdsourcing platforms offer a practical and cost-effective alternative for large datasets when individual annotation is impractical. In addition, these platforms gather labels from multiple labelers. Still, traditional multiple-annotator methods must account for the varying levels of expertise and the noise introduced by unreliable outputs, resulting in decreased performance. In addition, they assume a homogeneous behavior of the labelers across the input feature space, and independence constraints are imposed on outputs. We propose a Generalized Cross-Entropy-based framework using Chained Deep Learning (GCECDL) to code each annotator’s non-stationary patterns regarding the input space while preserving the inter-dependencies among experts through a chained deep learning approach. Experimental results devoted to multiple-annotator classification tasks on several well-known datasets demonstrate that our GCECDL can achieve robust predictive properties, outperforming state-of-the-art algorithms by combining the power of deep learning with a noise-robust loss function to deal with noisy labels. Moreover, network self-regularization is achieved by estimating each labeler’s reliability within the chained approach. Lastly, visual inspection and relevance analysis experiments are conducted to reveal the non-stationary coding of our method. In a nutshell, GCEDL weights reliable labelers as a function of each input sample and achieves suitable discrimination performance with preserved interpretability regarding each annotator’s trustworthiness estimation. |
|---|