Análisis de calibración en modelos de aprendizaje de máquina cuántico

El análisis de calibración de modelos de aprendizaje de máquina cobra gran importancia en distintos contextos como evaluación del riesgo, diagnósticos y sistemas críticos para la seguridad, donde hay decisiones influenciadas por las predicciones de los modelos. El área del aprendizaje de máquina cuá...

Full description

Autores:
Amaya Cruz, Glenn Harry
Tipo de recurso:
Fecha de publicación:
2023
Institución:
Universidad Nacional de Colombia
Repositorio:
Universidad Nacional de Colombia
Idioma:
spa
OAI Identifier:
oai:repositorio.unal.edu.co:unal/83732
Acceso en línea:
https://repositorio.unal.edu.co/handle/unal/83732
https://repositorio.unal.edu.co/
Palabra clave:
620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingeniería
620 - Ingeniería y operaciones afines::621 - Física aplicada
Evaluación de riesgos
Teoría del campo cuántico
Risk assessment
Quantum field theory
Aprendizaje de máquina
Aprendizaje de máquina cuántico
Calibración
Análisis de confianza
Machine learning
Quantum machine learning
Calibration
Confident analysis
Rights
openAccess
License
Atribución-NoComercial 4.0 Internacional
id UNACIONAL2_3d4b799bb757df1298e20caa9d64111d
oai_identifier_str oai:repositorio.unal.edu.co:unal/83732
network_acronym_str UNACIONAL2
network_name_str Universidad Nacional de Colombia
repository_id_str
dc.title.spa.fl_str_mv Análisis de calibración en modelos de aprendizaje de máquina cuántico
dc.title.translated.eng.fl_str_mv Calibration analysis in quantum machine learning models
title Análisis de calibración en modelos de aprendizaje de máquina cuántico
spellingShingle Análisis de calibración en modelos de aprendizaje de máquina cuántico
620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingeniería
620 - Ingeniería y operaciones afines::621 - Física aplicada
Evaluación de riesgos
Teoría del campo cuántico
Risk assessment
Quantum field theory
Aprendizaje de máquina
Aprendizaje de máquina cuántico
Calibración
Análisis de confianza
Machine learning
Quantum machine learning
Calibration
Confident analysis
title_short Análisis de calibración en modelos de aprendizaje de máquina cuántico
title_full Análisis de calibración en modelos de aprendizaje de máquina cuántico
title_fullStr Análisis de calibración en modelos de aprendizaje de máquina cuántico
title_full_unstemmed Análisis de calibración en modelos de aprendizaje de máquina cuántico
title_sort Análisis de calibración en modelos de aprendizaje de máquina cuántico
dc.creator.fl_str_mv Amaya Cruz, Glenn Harry
dc.contributor.advisor.none.fl_str_mv Gonzalez Osorio, Fabio Augusto
Toledo Cortés, Santiago
dc.contributor.author.none.fl_str_mv Amaya Cruz, Glenn Harry
dc.contributor.researchgroup.spa.fl_str_mv Mindlab
dc.subject.ddc.spa.fl_str_mv 620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingeniería
620 - Ingeniería y operaciones afines::621 - Física aplicada
topic 620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingeniería
620 - Ingeniería y operaciones afines::621 - Física aplicada
Evaluación de riesgos
Teoría del campo cuántico
Risk assessment
Quantum field theory
Aprendizaje de máquina
Aprendizaje de máquina cuántico
Calibración
Análisis de confianza
Machine learning
Quantum machine learning
Calibration
Confident analysis
dc.subject.lemb.spa.fl_str_mv Evaluación de riesgos
Teoría del campo cuántico
dc.subject.lemb.eng.fl_str_mv Risk assessment
Quantum field theory
dc.subject.proposal.spa.fl_str_mv Aprendizaje de máquina
Aprendizaje de máquina cuántico
Calibración
Análisis de confianza
dc.subject.proposal.eng.fl_str_mv Machine learning
Quantum machine learning
Calibration
Confident analysis
description El análisis de calibración de modelos de aprendizaje de máquina cobra gran importancia en distintos contextos como evaluación del riesgo, diagnósticos y sistemas críticos para la seguridad, donde hay decisiones influenciadas por las predicciones de los modelos. El área del aprendizaje de máquina cuántico ha recibido una mayor atención en los últimos años, en particular, se han desarrollado modelos que obtienen resultados competitivos en tareas de clasificación y regresión a comparación con métodos ampliamente utilizados. No obstante, las propiedades de este tipo de clasificadores en términos de calibración no han sido exploradas en la literatura. Por esta razón, en el presente trabajo se realiza un estudio de las propiedades de calibración que tienen algunos modelos de aprendizaje de máquina cuántico frente a modelos ampliamente usados en la literatura como máquinas de soporte vectorial, árboles de decisión, regresión logística, entre otros para tareas de clasificación binaria y de múltiples clases. Adicionalmente, se realiza un experimento para explorar el efecto de algunos clasificadores cuánticos en combinación con una red neuronal. Los resultados experimentales muestran que algunos de los clasificadores cuánticos analizados tienen un rendimiento competitivo e incluso mejor en métricas de calibración y las tareas de clasificación. (texto tomado de la fuente)
publishDate 2023
dc.date.accessioned.none.fl_str_mv 2023-04-18T22:42:58Z
dc.date.available.none.fl_str_mv 2023-04-18T22:42:58Z
dc.date.issued.none.fl_str_mv 2023
dc.type.spa.fl_str_mv Trabajo de grado - Maestría
dc.type.driver.spa.fl_str_mv info:eu-repo/semantics/masterThesis
dc.type.version.spa.fl_str_mv info:eu-repo/semantics/acceptedVersion
dc.type.content.spa.fl_str_mv Text
dc.type.redcol.spa.fl_str_mv http://purl.org/redcol/resource_type/TM
status_str acceptedVersion
dc.identifier.uri.none.fl_str_mv https://repositorio.unal.edu.co/handle/unal/83732
dc.identifier.instname.spa.fl_str_mv Universidad Nacional de Colombia
dc.identifier.reponame.spa.fl_str_mv Repositorio Institucional Universidad Nacional de Colombia
dc.identifier.repourl.spa.fl_str_mv https://repositorio.unal.edu.co/
url https://repositorio.unal.edu.co/handle/unal/83732
https://repositorio.unal.edu.co/
identifier_str_mv Universidad Nacional de Colombia
Repositorio Institucional Universidad Nacional de Colombia
dc.language.iso.spa.fl_str_mv spa
language spa
dc.relation.references.spa.fl_str_mv Ayhan, Murat S. ; Berens, Philipp: Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks. In: Medical Imaging with Deep Learning (MIDL), 2018, S. 1–9
Aha, D ; Kibler, Dennis: Instance-based prediction of heart-disease presence with the Cleveland database. University of California 3 (1988), Nr. 1, S. 3–2
Bastola, S. ; Ishidaira, H. ; Takeuchi, K.: Regionalisation of hydrological model parameters under parameter uncertainty: A case study involving TOPMODEL and basins across the globe. Journal of Hydrology 357 (2008), Nr. 3-4, S. 188–206
Beaudouin, R. ; Monod, G. ; Ginot, V.: Selecting parameters for calibration via sensitivity analysis: An individual-based model of mosquitofish population dynamics. Ecological Modelling 218 (2008), Nr. 1-2, S. 29–48
Brier, Glenn W.: Verification of forecast expressed in terms of probability. Monthly Weather Review 78 (1950), jan, Nr. 1, S. 1–3. – ISSN 0027–0644
Bröcker, Jochen ; Smith, Leonard A.: Increasing the reliability of reliability diagrams. Weather and Forecasting 22 (2007), jun, Nr. 3, S. 651–661. – ISSN 08828156
Biamonte, Jacob ; Wittek, Peter ; Pancotti, Nicola ; Rebentrost, Patrick ; Wiebe, Nathan ; Lloyd, Seth: Quantum machine learning. Nature 549 (2017), Nr. 7671, S. 195–202
Chollet, Fran¸cois ; others. Keras. https://keras.io. 2015
Charoenpanyanet, A.: Modeling Anopheles mosquito density spatial and seasonal variations using remotely sensed imagery and statistical methods. International Journal of Geoinformatics 13 (2017), Nr. 1, S. 35–47
Carneiro, G. ; Zorron Cheng Tao Pu, L. ; Singh, R. ; Burt, A.: Deep learning uncertainty and confidence calibration for the five-class polyp classification from colonoscopy. Medical Image Analysis 62 (2020)
Degroot, M. D. ; Fienberg, Stephen E. The comparison and evaluation of forecasters. mar 1982
Dua, Dheeru ; Graff, Casey. UCI Machine Learning Repository. 2017
Demiroz, G ; Govenir, HA ; Ilter, N: Learning differential diagnosis of eryhematosquamous diseases using voting feature intervals. Artificial Intelligence in Medicine 13 (1998), Nr. 3, S. 147–165
Diego Hernando, Useche R.: Quantum measurement learning for medical image classification. (2022)
Detrano, Robert ; Janosi, Andras ; Steinbrunn, Walter ; Pfisterer, Matthias ; Schmid, Johann-Jakob ; Sandhu, Sarbjit ; Guppy, Kern H. ; Lee, Stella ; Froelicher, Victor: International application of a new probability algorithm for the diagnosis of coronary artery disease. The American journal of cardiology 64 (1989), Nr. 5, S. 304–310
Dormann, Carsten F.: Calibration of probability predictions from machine-learning and statistical models. Global Ecology and Biogeography 29 (2020), apr, Nr. 4, S. 760–765. – ISSN 14668238
Deng, Y. ; Yang, B.-R. ; Luo, J.-W. ; Du, G.-X. ; Luo, L.-P.: DTI-based radiomics signature for the detection of early diabetic kidney damage. Abdominal Radiology 45 (2020), Nr. 8, S. 2526–2531
Feng, Runhai: Improving uncertainty analysis in well log classification by machine learning with a scaling algorithm. Journal of Petroleum Science and Engineering 196 (2021). – ISSN 09204105
Foster, Dean P. ; Vohra, Rakesh V.: Asymptotic calibration. Biometrika 85 (1998), Nr. 2, S. 379–390. – ISSN 00063444
Galindo, Y. ; De Cicco, M. ; Quiles, M.G. ; Lorena, A.C.: Monitoring Night Skies with Deep Learning. Bd. 1332. 2020 460–468 Seiten. – ISBN 9783030638191
González, Fabio A. ; Gallego, Alejandro ; Toledo-Cortés, Santiago ; Vargas-Calderón, Vladimir: Learning with Density Matrices and Random Features. arXiv preprint arXiv:2102.04394 (2021)
Gallego-Mejia, Joseph ; Bustos-Brinez, Oscar ; Gonzalez, Fabio: InQMAD: Incremental Quantum Measurement Anomaly Detection. arXiv preprint arXiv:2210.05061 (2022)
Glasser, Ivan ; Pancotti, Nicola ; Ignacio Cirac, J.: From Probabilistic Graphical Models to Generalized Tensor Networks for Supervised Learning. IEEE Access 8 (2020), S. 68169–68182. – ISSN 21693536
Guo, Chuan ; Pleiss, Geoff ; Sun, Yu ; Weinberger, Kilian Q. On calibration of modern neural networks. 2017
Gupta, Kartik ; Rahimi, Amir ; Ajanthan, Thalaiyasingam ; Mensink, Thomas ; Sminchisescu, Cristian ; Hartley, Richard: Calibration of neural networks using splines. arXiv preprint arXiv:2006.12800 (2020), 6
Guan, Yawen ; Sampson, Christian ; Tucker, J. D. ; Chang, Won ; Mondal, Anirban ; Haran, Murali ; Sulsky, Deborah: Computer Model Calibration Based on ImageWarping Metrics: An Application for Sea Ice Deformation. Journal of Agricultural, Biological, and Environmental Statistics 24 (2019), Nr. 3, S. 444–463. – ISSN 15372693
Ghoshal, Biraja ; Tucker, Allan: On calibrated model uncertainty in deep learning. arXiv preprint arXiv:2206.07795 (2022)
González, Fabio A. ; Vargas-Calderón, Vladimir ; Vinck-Posada, Herbert: Classification with Quantum Measurements. Journal of the Physical Society of Japan 90 (2021), Nr. 4, S. 044002
Higdon, Dave ; Gattiker, James ; Williams, Brian ; Rightley, Maria: Computer model calibration using high-dimensional output. Journal of the American Statistical Association 103 (2008), Nr. 482, S. 570–583. – ISSN 01621459
Iliyasu, Abdullah M. ; Fatichah, Chastine: A quantum hybrid PSO combined with fuzzy k-NN approach to feature selection and cell classification in cervical cancer detection. Sensors (Switzerland) 17 (2017), Nr. 12. – ISSN 14248220
Jensen, M.H. ; Jørgensen, D.R. ; Jalaboi, R. ; Hansen, M.E. ; Olsen, M.A.: Improving uncertainty estimation in convolutional neural networks using inter-rater agreement. Bd. 11767 LNCS. 2019 540–548 Seiten. – ISBN 9783030322502
Jiang, Xiaoqian ; Osl, Melanie ; Kim, Jihoon ; Ohno-Machado, Lucila: Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Association 19 (2012), mar, Nr. 2, S. 263–274. – ISSN 10675027
Kuleshov, Volodymyr ; Ermon, Stefano: Estimating uncertainty online against an adversary. In: 31st AAAI Conference on Artificial Intelligence, AAAI 2017 , 2017, S. 2110–2116
Krizhevsky, Alex ; Hinton, Geoffrey ; others: Learning multiple layers of features from tiny images. (2009)
Kuhn, Max ; Johnson, Kjell: Applied predictive modeling. Bd. 26. Springer, 2013
Kuleshov, Volodymyr ; Liang, Percy: Calibrated structured prediction. In: Advances in Neural Information Processing Systems Bd. 2015-Janua. Bd. 2015-Janua, 2015. ISSN 10495258, S. 3474–3482
Kumar, Ananya ; Liang, Percy S. ; Ma, Tengyu: Verified uncertainty calibration. Advances in Neural Information Processing Systems 32 (2019)
Kull, Meelis ; Perello Nieto, Miquel ; K¨angsepp, Markus ; Silva Filho, Telmo ; Song, Hao ; Flach, Peter. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. 2019
Müller, R. ; Kornblith, S. ; Hinton, G.: When does label smoothing help? In: Advances in Neural Information Processing Systems Bd. 32. Bd. 32, 2019
Naeini, Mahdi P. ; Cooper, Gregory ; Hauskrecht, Milos: Obtaining well calibrated probabilities using bayesian binning. In: Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015
Nixon, Jeremy ; Dusenberry, Michael W. ; Zhang, Linchuan ; Jerfel, Ghassen ; Tran, Dustin: Measuring Calibration in Deep Learning. 2 (2019), Nr. 7
Niculescu-Mizil, Alexandru ; Caruana, Rich: Predicting good probabilities with supervised learning. In: ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning, 2005. – ISBN 1595931805, S. 625–632
Nguyen, Khanh ; O’Connor, Brendan: Posterior calibration and exploratory analysis for natural language processing models. In: Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing, 2015. – ISBN 9781941643327, S. 1587–1598
Posocco, Nicolas ; Bonnefoy, Antoine: Estimating Expected Calibration Errors. In: International Conference on Artificial Neural Networks Springer, 2021, S. 139–150
Peleg, K.: Fast fourier transform based calibration in remote sensing. International Journal of Remote Sensing 19 (1998), Nr. 12, S. 2301–2315
Pearce, Jennie ; Ferrier, Simon: Evaluating the predictive performance of habitat models developed using logistic regression. Ecological Modelling 133 (2000), Nr. 3, S. 225–245. – ISSN 03043800
Platt, John ; Others: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10 (1999), Nr. 3, S. 61–74
Pedregosa, F. ; Varoquaux, G. ; Gramfort, A. ; Michel, V. ; Thirion, B. ; Grisel, O. ; Blondel, M. ; Prettenhofer, P. ; Weiss, R. ; Dubourg, V. ; Vanderplas, J. ; Passos, A. ; Cournapeau, D. ; Brucher, M. ; Perrot, M. ; Duchesnay, E.: Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), S. 2825–2830
Sergioli, Giuseppe ; Militello, Carmelo ; Rundo, Leonardo ; Minafra, Luigi ; Torrisi, Filippo ; Russo, Giorgio ; Chow, Keng L. ; Giuntini, Roberto: A quantum-inspired classifier for clonogenic assay evaluations. Scientific Reports 11 (2021), Nr. 1, S. 2830. – ISBN 0123456789
Schuld, Maria ; Sinayskiy, Ilya ; Petruccione, Francesco: An introduction to quantum machine learning. Contemporary Physics 56 (2015), Nr. 2, S. 172–185. – ISSN 13665812
Toledo-Cortés, Santiago ; Useche, Diego H. ; González, Fabio A.: Prostate Tissue Grading with Deep Quantum Measurement Ordinal Regression. arXiv preprint arXiv:2103.03188 (2021)
Torres-Meza, M.d.J. ; Báez-González, A.D. ; Maciel-Pérez, L.H. ; Quezada-Guzmán, E. ; Sierra-Tristán, J.S.: GIS-based modeling of the geographic distribution of Quercus emoryi Torr. (Fagaceae) in México and identification of significant environmental factors influencing the species’ distribution. Ecological Modelling 220 (2009), Nr. 24, S. 3599– 3611
Vaicenavicius, Juozas ; Widmann, David ; Andersson, Carl ; Lindsten, Fredrik ; Roll, Jacob ; Sch¨on, Thomas: Evaluating model calibration in classification. In: The 22nd International Conference on Artificial Intelligence and Statistics PMLR, 2019, S. 3459– 3467
Widmann, David ; Lindsten, Fredrik ; Zachariah, Dave: Calibration tests in multiclass classification: A unifying framework. Advances in Neural Information Processing Systems 32 (2019)
Zadrozny, Bianca ; Elkan, Charles: Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers. Icml (2001), S. 1–8. ISBN 1–55860–778–1
Zadrozny, Bianca ; Elkan, Charles: Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2002. – ISBN 158113567X, S. 694–699
LeCun, Yann ; Bottou, Léon ; Bengio, Yoshua ; Haffner, Patrick: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86 (1998), Nr. 11, S. 2278–2324
dc.rights.coar.fl_str_mv http://purl.org/coar/access_right/c_abf2
dc.rights.license.spa.fl_str_mv Atribución-NoComercial 4.0 Internacional
dc.rights.uri.spa.fl_str_mv http://creativecommons.org/licenses/by-nc/4.0/
dc.rights.accessrights.spa.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv Atribución-NoComercial 4.0 Internacional
http://creativecommons.org/licenses/by-nc/4.0/
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.extent.spa.fl_str_mv xiii, 53 páginas
dc.format.mimetype.spa.fl_str_mv application/pdf
dc.publisher.spa.fl_str_mv Universidad Nacional de Colombia
dc.publisher.program.spa.fl_str_mv Bogotá - Ingeniería - Maestría en Ingeniería - Ingeniería de Sistemas y Computación
dc.publisher.faculty.spa.fl_str_mv Facultad de Ingeniería
dc.publisher.place.spa.fl_str_mv Bogotá,Colombia
dc.publisher.branch.spa.fl_str_mv Universidad Nacional de Colombia - Sede Bogotá
institution Universidad Nacional de Colombia
bitstream.url.fl_str_mv https://repositorio.unal.edu.co/bitstream/unal/83732/3/1022435940.2023.pdf.jpg
https://repositorio.unal.edu.co/bitstream/unal/83732/1/license.txt
https://repositorio.unal.edu.co/bitstream/unal/83732/2/1022435940.2023.pdf
bitstream.checksum.fl_str_mv ace460b546c6a0a85559b2564debbd49
eb34b1cf90b7e1103fc9dfd26be24b4a
49cf69b064dbf288134a811a13560c0d
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
MD5
repository.name.fl_str_mv Repositorio Institucional Universidad Nacional de Colombia
repository.mail.fl_str_mv repositorio_nal@unal.edu.co
_version_ 1814090248470134784
spelling Atribución-NoComercial 4.0 Internacionalhttp://creativecommons.org/licenses/by-nc/4.0/info:eu-repo/semantics/openAccesshttp://purl.org/coar/access_right/c_abf2Gonzalez Osorio, Fabio Augusto0e9d70b5c1d7448338ca4467ccb27e59Toledo Cortés, Santiagoaacc1c99e2c7e404d2f99a7a954b57c8Amaya Cruz, Glenn Harryb0536d7a878b2132355a7f85bc1c017bMindlab2023-04-18T22:42:58Z2023-04-18T22:42:58Z2023https://repositorio.unal.edu.co/handle/unal/83732Universidad Nacional de ColombiaRepositorio Institucional Universidad Nacional de Colombiahttps://repositorio.unal.edu.co/El análisis de calibración de modelos de aprendizaje de máquina cobra gran importancia en distintos contextos como evaluación del riesgo, diagnósticos y sistemas críticos para la seguridad, donde hay decisiones influenciadas por las predicciones de los modelos. El área del aprendizaje de máquina cuántico ha recibido una mayor atención en los últimos años, en particular, se han desarrollado modelos que obtienen resultados competitivos en tareas de clasificación y regresión a comparación con métodos ampliamente utilizados. No obstante, las propiedades de este tipo de clasificadores en términos de calibración no han sido exploradas en la literatura. Por esta razón, en el presente trabajo se realiza un estudio de las propiedades de calibración que tienen algunos modelos de aprendizaje de máquina cuántico frente a modelos ampliamente usados en la literatura como máquinas de soporte vectorial, árboles de decisión, regresión logística, entre otros para tareas de clasificación binaria y de múltiples clases. Adicionalmente, se realiza un experimento para explorar el efecto de algunos clasificadores cuánticos en combinación con una red neuronal. Los resultados experimentales muestran que algunos de los clasificadores cuánticos analizados tienen un rendimiento competitivo e incluso mejor en métricas de calibración y las tareas de clasificación. (texto tomado de la fuente)Calibration of machine learning models is of great importance in different contexts such as risk assessment, diagnostics, and safety-critical systems, in which decisions are influenced by model predictions. The area of quantum machine learning has received an increased attention in recent years, in particular, models have been developed that obtain competitive results in classification and regression tasks compared to widely used methods. However, the properties of this type of classifiers in terms of calibration have not been explored in the literature. As a result, in this work a study of the properties of calibration is conducted for recent quantum machine learning models in comparison to state-of-the-art models such as support vector machines, decisions trees, logistic regression, and others for binary and multiclass classification tasks. Moreover, an experiment to explore the effect of some quantum classifiers in combination with a neural network is made. The experimental results show that some of the analyzed quantum classifiers have competitive and even better performance in calibration metrics and the classification tasks.MaestríaMagíster en Ingeniería - Ingeniería de Sistemas y ComputaciónSistemas Inteligentesxiii, 53 páginasapplication/pdfspaUniversidad Nacional de ColombiaBogotá - Ingeniería - Maestría en Ingeniería - Ingeniería de Sistemas y ComputaciónFacultad de IngenieríaBogotá,ColombiaUniversidad Nacional de Colombia - Sede Bogotá620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingeniería620 - Ingeniería y operaciones afines::621 - Física aplicadaEvaluación de riesgosTeoría del campo cuánticoRisk assessmentQuantum field theoryAprendizaje de máquinaAprendizaje de máquina cuánticoCalibraciónAnálisis de confianzaMachine learningQuantum machine learningCalibrationConfident analysisAnálisis de calibración en modelos de aprendizaje de máquina cuánticoCalibration analysis in quantum machine learning modelsTrabajo de grado - Maestríainfo:eu-repo/semantics/masterThesisinfo:eu-repo/semantics/acceptedVersionTexthttp://purl.org/redcol/resource_type/TMAyhan, Murat S. ; Berens, Philipp: Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks. In: Medical Imaging with Deep Learning (MIDL), 2018, S. 1–9Aha, D ; Kibler, Dennis: Instance-based prediction of heart-disease presence with the Cleveland database. University of California 3 (1988), Nr. 1, S. 3–2Bastola, S. ; Ishidaira, H. ; Takeuchi, K.: Regionalisation of hydrological model parameters under parameter uncertainty: A case study involving TOPMODEL and basins across the globe. Journal of Hydrology 357 (2008), Nr. 3-4, S. 188–206Beaudouin, R. ; Monod, G. ; Ginot, V.: Selecting parameters for calibration via sensitivity analysis: An individual-based model of mosquitofish population dynamics. Ecological Modelling 218 (2008), Nr. 1-2, S. 29–48Brier, Glenn W.: Verification of forecast expressed in terms of probability. Monthly Weather Review 78 (1950), jan, Nr. 1, S. 1–3. – ISSN 0027–0644Bröcker, Jochen ; Smith, Leonard A.: Increasing the reliability of reliability diagrams. Weather and Forecasting 22 (2007), jun, Nr. 3, S. 651–661. – ISSN 08828156Biamonte, Jacob ; Wittek, Peter ; Pancotti, Nicola ; Rebentrost, Patrick ; Wiebe, Nathan ; Lloyd, Seth: Quantum machine learning. Nature 549 (2017), Nr. 7671, S. 195–202Chollet, Fran¸cois ; others. Keras. https://keras.io. 2015Charoenpanyanet, A.: Modeling Anopheles mosquito density spatial and seasonal variations using remotely sensed imagery and statistical methods. International Journal of Geoinformatics 13 (2017), Nr. 1, S. 35–47Carneiro, G. ; Zorron Cheng Tao Pu, L. ; Singh, R. ; Burt, A.: Deep learning uncertainty and confidence calibration for the five-class polyp classification from colonoscopy. Medical Image Analysis 62 (2020)Degroot, M. D. ; Fienberg, Stephen E. The comparison and evaluation of forecasters. mar 1982Dua, Dheeru ; Graff, Casey. UCI Machine Learning Repository. 2017Demiroz, G ; Govenir, HA ; Ilter, N: Learning differential diagnosis of eryhematosquamous diseases using voting feature intervals. Artificial Intelligence in Medicine 13 (1998), Nr. 3, S. 147–165Diego Hernando, Useche R.: Quantum measurement learning for medical image classification. (2022)Detrano, Robert ; Janosi, Andras ; Steinbrunn, Walter ; Pfisterer, Matthias ; Schmid, Johann-Jakob ; Sandhu, Sarbjit ; Guppy, Kern H. ; Lee, Stella ; Froelicher, Victor: International application of a new probability algorithm for the diagnosis of coronary artery disease. The American journal of cardiology 64 (1989), Nr. 5, S. 304–310Dormann, Carsten F.: Calibration of probability predictions from machine-learning and statistical models. Global Ecology and Biogeography 29 (2020), apr, Nr. 4, S. 760–765. – ISSN 14668238Deng, Y. ; Yang, B.-R. ; Luo, J.-W. ; Du, G.-X. ; Luo, L.-P.: DTI-based radiomics signature for the detection of early diabetic kidney damage. Abdominal Radiology 45 (2020), Nr. 8, S. 2526–2531Feng, Runhai: Improving uncertainty analysis in well log classification by machine learning with a scaling algorithm. Journal of Petroleum Science and Engineering 196 (2021). – ISSN 09204105Foster, Dean P. ; Vohra, Rakesh V.: Asymptotic calibration. Biometrika 85 (1998), Nr. 2, S. 379–390. – ISSN 00063444Galindo, Y. ; De Cicco, M. ; Quiles, M.G. ; Lorena, A.C.: Monitoring Night Skies with Deep Learning. Bd. 1332. 2020 460–468 Seiten. – ISBN 9783030638191González, Fabio A. ; Gallego, Alejandro ; Toledo-Cortés, Santiago ; Vargas-Calderón, Vladimir: Learning with Density Matrices and Random Features. arXiv preprint arXiv:2102.04394 (2021)Gallego-Mejia, Joseph ; Bustos-Brinez, Oscar ; Gonzalez, Fabio: InQMAD: Incremental Quantum Measurement Anomaly Detection. arXiv preprint arXiv:2210.05061 (2022)Glasser, Ivan ; Pancotti, Nicola ; Ignacio Cirac, J.: From Probabilistic Graphical Models to Generalized Tensor Networks for Supervised Learning. IEEE Access 8 (2020), S. 68169–68182. – ISSN 21693536Guo, Chuan ; Pleiss, Geoff ; Sun, Yu ; Weinberger, Kilian Q. On calibration of modern neural networks. 2017Gupta, Kartik ; Rahimi, Amir ; Ajanthan, Thalaiyasingam ; Mensink, Thomas ; Sminchisescu, Cristian ; Hartley, Richard: Calibration of neural networks using splines. arXiv preprint arXiv:2006.12800 (2020), 6Guan, Yawen ; Sampson, Christian ; Tucker, J. D. ; Chang, Won ; Mondal, Anirban ; Haran, Murali ; Sulsky, Deborah: Computer Model Calibration Based on ImageWarping Metrics: An Application for Sea Ice Deformation. Journal of Agricultural, Biological, and Environmental Statistics 24 (2019), Nr. 3, S. 444–463. – ISSN 15372693Ghoshal, Biraja ; Tucker, Allan: On calibrated model uncertainty in deep learning. arXiv preprint arXiv:2206.07795 (2022)González, Fabio A. ; Vargas-Calderón, Vladimir ; Vinck-Posada, Herbert: Classification with Quantum Measurements. Journal of the Physical Society of Japan 90 (2021), Nr. 4, S. 044002Higdon, Dave ; Gattiker, James ; Williams, Brian ; Rightley, Maria: Computer model calibration using high-dimensional output. Journal of the American Statistical Association 103 (2008), Nr. 482, S. 570–583. – ISSN 01621459Iliyasu, Abdullah M. ; Fatichah, Chastine: A quantum hybrid PSO combined with fuzzy k-NN approach to feature selection and cell classification in cervical cancer detection. Sensors (Switzerland) 17 (2017), Nr. 12. – ISSN 14248220Jensen, M.H. ; Jørgensen, D.R. ; Jalaboi, R. ; Hansen, M.E. ; Olsen, M.A.: Improving uncertainty estimation in convolutional neural networks using inter-rater agreement. Bd. 11767 LNCS. 2019 540–548 Seiten. – ISBN 9783030322502Jiang, Xiaoqian ; Osl, Melanie ; Kim, Jihoon ; Ohno-Machado, Lucila: Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Association 19 (2012), mar, Nr. 2, S. 263–274. – ISSN 10675027Kuleshov, Volodymyr ; Ermon, Stefano: Estimating uncertainty online against an adversary. In: 31st AAAI Conference on Artificial Intelligence, AAAI 2017 , 2017, S. 2110–2116Krizhevsky, Alex ; Hinton, Geoffrey ; others: Learning multiple layers of features from tiny images. (2009)Kuhn, Max ; Johnson, Kjell: Applied predictive modeling. Bd. 26. Springer, 2013Kuleshov, Volodymyr ; Liang, Percy: Calibrated structured prediction. In: Advances in Neural Information Processing Systems Bd. 2015-Janua. Bd. 2015-Janua, 2015. ISSN 10495258, S. 3474–3482Kumar, Ananya ; Liang, Percy S. ; Ma, Tengyu: Verified uncertainty calibration. Advances in Neural Information Processing Systems 32 (2019)Kull, Meelis ; Perello Nieto, Miquel ; K¨angsepp, Markus ; Silva Filho, Telmo ; Song, Hao ; Flach, Peter. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. 2019Müller, R. ; Kornblith, S. ; Hinton, G.: When does label smoothing help? In: Advances in Neural Information Processing Systems Bd. 32. Bd. 32, 2019Naeini, Mahdi P. ; Cooper, Gregory ; Hauskrecht, Milos: Obtaining well calibrated probabilities using bayesian binning. In: Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015Nixon, Jeremy ; Dusenberry, Michael W. ; Zhang, Linchuan ; Jerfel, Ghassen ; Tran, Dustin: Measuring Calibration in Deep Learning. 2 (2019), Nr. 7Niculescu-Mizil, Alexandru ; Caruana, Rich: Predicting good probabilities with supervised learning. In: ICML 2005 - Proceedings of the 22nd International Conference on Machine Learning, 2005. – ISBN 1595931805, S. 625–632Nguyen, Khanh ; O’Connor, Brendan: Posterior calibration and exploratory analysis for natural language processing models. In: Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing, 2015. – ISBN 9781941643327, S. 1587–1598Posocco, Nicolas ; Bonnefoy, Antoine: Estimating Expected Calibration Errors. In: International Conference on Artificial Neural Networks Springer, 2021, S. 139–150Peleg, K.: Fast fourier transform based calibration in remote sensing. International Journal of Remote Sensing 19 (1998), Nr. 12, S. 2301–2315Pearce, Jennie ; Ferrier, Simon: Evaluating the predictive performance of habitat models developed using logistic regression. Ecological Modelling 133 (2000), Nr. 3, S. 225–245. – ISSN 03043800Platt, John ; Others: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10 (1999), Nr. 3, S. 61–74Pedregosa, F. ; Varoquaux, G. ; Gramfort, A. ; Michel, V. ; Thirion, B. ; Grisel, O. ; Blondel, M. ; Prettenhofer, P. ; Weiss, R. ; Dubourg, V. ; Vanderplas, J. ; Passos, A. ; Cournapeau, D. ; Brucher, M. ; Perrot, M. ; Duchesnay, E.: Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), S. 2825–2830Sergioli, Giuseppe ; Militello, Carmelo ; Rundo, Leonardo ; Minafra, Luigi ; Torrisi, Filippo ; Russo, Giorgio ; Chow, Keng L. ; Giuntini, Roberto: A quantum-inspired classifier for clonogenic assay evaluations. Scientific Reports 11 (2021), Nr. 1, S. 2830. – ISBN 0123456789Schuld, Maria ; Sinayskiy, Ilya ; Petruccione, Francesco: An introduction to quantum machine learning. Contemporary Physics 56 (2015), Nr. 2, S. 172–185. – ISSN 13665812Toledo-Cortés, Santiago ; Useche, Diego H. ; González, Fabio A.: Prostate Tissue Grading with Deep Quantum Measurement Ordinal Regression. arXiv preprint arXiv:2103.03188 (2021)Torres-Meza, M.d.J. ; Báez-González, A.D. ; Maciel-Pérez, L.H. ; Quezada-Guzmán, E. ; Sierra-Tristán, J.S.: GIS-based modeling of the geographic distribution of Quercus emoryi Torr. (Fagaceae) in México and identification of significant environmental factors influencing the species’ distribution. Ecological Modelling 220 (2009), Nr. 24, S. 3599– 3611Vaicenavicius, Juozas ; Widmann, David ; Andersson, Carl ; Lindsten, Fredrik ; Roll, Jacob ; Sch¨on, Thomas: Evaluating model calibration in classification. In: The 22nd International Conference on Artificial Intelligence and Statistics PMLR, 2019, S. 3459– 3467Widmann, David ; Lindsten, Fredrik ; Zachariah, Dave: Calibration tests in multiclass classification: A unifying framework. Advances in Neural Information Processing Systems 32 (2019)Zadrozny, Bianca ; Elkan, Charles: Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers. Icml (2001), S. 1–8. ISBN 1–55860–778–1Zadrozny, Bianca ; Elkan, Charles: Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2002. – ISBN 158113567X, S. 694–699LeCun, Yann ; Bottou, Léon ; Bengio, Yoshua ; Haffner, Patrick: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86 (1998), Nr. 11, S. 2278–2324EstudiantesInvestigadoresTHUMBNAIL1022435940.2023.pdf.jpg1022435940.2023.pdf.jpgGenerated Thumbnailimage/jpeg4143https://repositorio.unal.edu.co/bitstream/unal/83732/3/1022435940.2023.pdf.jpgace460b546c6a0a85559b2564debbd49MD53LICENSElicense.txtlicense.txttext/plain; charset=utf-85879https://repositorio.unal.edu.co/bitstream/unal/83732/1/license.txteb34b1cf90b7e1103fc9dfd26be24b4aMD51ORIGINAL1022435940.2023.pdf1022435940.2023.pdfTesis de Maestría en Ingeniería de Sistemasapplication/pdf849061https://repositorio.unal.edu.co/bitstream/unal/83732/2/1022435940.2023.pdf49cf69b064dbf288134a811a13560c0dMD52unal/83732oai:repositorio.unal.edu.co:unal/837322023-08-02 23:03:57.201Repositorio Institucional Universidad Nacional de Colombiarepositorio_nal@unal.edu.coUEFSVEUgMS4gVMOJUk1JTk9TIERFIExBIExJQ0VOQ0lBIFBBUkEgUFVCTElDQUNJw5NOIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KCkxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgbG9zIGRlcmVjaG9zIHBhdHJpbW9uaWFsZXMgZGUgYXV0b3IsIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEsIGxpbWl0YWRhIHkgZ3JhdHVpdGEgc29icmUgbGEgb2JyYSBxdWUgc2UgaW50ZWdyYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsLCBiYWpvIGxvcyBzaWd1aWVudGVzIHTDqXJtaW5vczoKCgphKQlMb3MgYXV0b3JlcyB5L28gbG9zIHRpdHVsYXJlcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBzb2JyZSBsYSBvYnJhIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEgcGFyYSByZWFsaXphciBsb3Mgc2lndWllbnRlcyBhY3RvcyBzb2JyZSBsYSBvYnJhOiBpKSByZXByb2R1Y2lyIGxhIG9icmEgZGUgbWFuZXJhIGRpZ2l0YWwsIHBlcm1hbmVudGUgbyB0ZW1wb3JhbCwgaW5jbHV5ZW5kbyBlbCBhbG1hY2VuYW1pZW50byBlbGVjdHLDs25pY28sIGFzw60gY29tbyBjb252ZXJ0aXIgZWwgZG9jdW1lbnRvIGVuIGVsIGN1YWwgc2UgZW5jdWVudHJhIGNvbnRlbmlkYSBsYSBvYnJhIGEgY3VhbHF1aWVyIG1lZGlvIG8gZm9ybWF0byBleGlzdGVudGUgYSBsYSBmZWNoYSBkZSBsYSBzdXNjcmlwY2nDs24gZGUgbGEgcHJlc2VudGUgbGljZW5jaWEsIHkgaWkpIGNvbXVuaWNhciBhbCBww7pibGljbyBsYSBvYnJhIHBvciBjdWFscXVpZXIgbWVkaW8gbyBwcm9jZWRpbWllbnRvLCBlbiBtZWRpb3MgYWzDoW1icmljb3MgbyBpbmFsw6FtYnJpY29zLCBpbmNsdXllbmRvIGxhIHB1ZXN0YSBhIGRpc3Bvc2ljacOzbiBlbiBhY2Nlc28gYWJpZXJ0by4gQWRpY2lvbmFsIGEgbG8gYW50ZXJpb3IsIGVsIGF1dG9yIHkvbyB0aXR1bGFyIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgcGFyYSBxdWUsIGVuIGxhIHJlcHJvZHVjY2nDs24geSBjb211bmljYWNpw7NuIGFsIHDDumJsaWNvIHF1ZSBsYSBVbml2ZXJzaWRhZCByZWFsaWNlIHNvYnJlIGxhIG9icmEsIGhhZ2EgbWVuY2nDs24gZGUgbWFuZXJhIGV4cHJlc2EgYWwgdGlwbyBkZSBsaWNlbmNpYSBDcmVhdGl2ZSBDb21tb25zIGJham8gbGEgY3VhbCBlbCBhdXRvciB5L28gdGl0dWxhciBkZXNlYSBvZnJlY2VyIHN1IG9icmEgYSBsb3MgdGVyY2Vyb3MgcXVlIGFjY2VkYW4gYSBkaWNoYSBvYnJhIGEgdHJhdsOpcyBkZWwgUmVwb3NpdG9yaW8gSW5zdGl0dWNpb25hbCwgY3VhbmRvIHNlYSBlbCBjYXNvLiBFbCBhdXRvciB5L28gdGl0dWxhciBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBwb2Ryw6EgZGFyIHBvciB0ZXJtaW5hZGEgbGEgcHJlc2VudGUgbGljZW5jaWEgbWVkaWFudGUgc29saWNpdHVkIGVsZXZhZGEgYSBsYSBEaXJlY2Npw7NuIE5hY2lvbmFsIGRlIEJpYmxpb3RlY2FzIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLiAKCmIpIAlMb3MgYXV0b3JlcyB5L28gdGl0dWxhcmVzIGRlIGxvcyBkZXJlY2hvcyBwYXRyaW1vbmlhbGVzIGRlIGF1dG9yIHNvYnJlIGxhIG9icmEgY29uZmllcmVuIGxhIGxpY2VuY2lhIHNlw7FhbGFkYSBlbiBlbCBsaXRlcmFsIGEpIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8gcG9yIGVsIHRpZW1wbyBkZSBwcm90ZWNjacOzbiBkZSBsYSBvYnJhIGVuIHRvZG9zIGxvcyBwYcOtc2VzIGRlbCBtdW5kbywgZXN0byBlcywgc2luIGxpbWl0YWNpw7NuIHRlcnJpdG9yaWFsIGFsZ3VuYS4KCmMpCUxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBtYW5pZmllc3RhbiBlc3RhciBkZSBhY3VlcmRvIGNvbiBxdWUgbGEgcHJlc2VudGUgbGljZW5jaWEgc2Ugb3RvcmdhIGEgdMOtdHVsbyBncmF0dWl0bywgcG9yIGxvIHRhbnRvLCByZW51bmNpYW4gYSByZWNpYmlyIGN1YWxxdWllciByZXRyaWJ1Y2nDs24gZWNvbsOzbWljYSBvIGVtb2x1bWVudG8gYWxndW5vIHBvciBsYSBwdWJsaWNhY2nDs24sIGRpc3RyaWJ1Y2nDs24sIGNvbXVuaWNhY2nDs24gcMO6YmxpY2EgeSBjdWFscXVpZXIgb3RybyB1c28gcXVlIHNlIGhhZ2EgZW4gbG9zIHTDqXJtaW5vcyBkZSBsYSBwcmVzZW50ZSBsaWNlbmNpYSB5IGRlIGxhIGxpY2VuY2lhIENyZWF0aXZlIENvbW1vbnMgY29uIHF1ZSBzZSBwdWJsaWNhLgoKZCkJUXVpZW5lcyBmaXJtYW4gZWwgcHJlc2VudGUgZG9jdW1lbnRvIGRlY2xhcmFuIHF1ZSBwYXJhIGxhIGNyZWFjacOzbiBkZSBsYSBvYnJhLCBubyBzZSBoYW4gdnVsbmVyYWRvIGxvcyBkZXJlY2hvcyBkZSBwcm9waWVkYWQgaW50ZWxlY3R1YWwsIGluZHVzdHJpYWwsIG1vcmFsZXMgeSBwYXRyaW1vbmlhbGVzIGRlIHRlcmNlcm9zLiBEZSBvdHJhIHBhcnRlLCAgcmVjb25vY2VuIHF1ZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhY3TDumEgY29tbyB1biB0ZXJjZXJvIGRlIGJ1ZW5hIGZlIHkgc2UgZW5jdWVudHJhIGV4ZW50YSBkZSBjdWxwYSBlbiBjYXNvIGRlIHByZXNlbnRhcnNlIGFsZ8O6biB0aXBvIGRlIHJlY2xhbWFjacOzbiBlbiBtYXRlcmlhIGRlIGRlcmVjaG9zIGRlIGF1dG9yIG8gcHJvcGllZGFkIGludGVsZWN0dWFsIGVuIGdlbmVyYWwuIFBvciBsbyB0YW50bywgbG9zIGZpcm1hbnRlcyAgYWNlcHRhbiBxdWUgY29tbyB0aXR1bGFyZXMgw7puaWNvcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciwgYXN1bWlyw6FuIHRvZGEgbGEgcmVzcG9uc2FiaWxpZGFkIGNpdmlsLCBhZG1pbmlzdHJhdGl2YSB5L28gcGVuYWwgcXVlIHB1ZWRhIGRlcml2YXJzZSBkZSBsYSBwdWJsaWNhY2nDs24gZGUgbGEgb2JyYS4gIAoKZikJQXV0b3JpemFuIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgaW5jbHVpciBsYSBvYnJhIGVuIGxvcyBhZ3JlZ2Fkb3JlcyBkZSBjb250ZW5pZG9zLCBidXNjYWRvcmVzIGFjYWTDqW1pY29zLCBtZXRhYnVzY2Fkb3Jlcywgw61uZGljZXMgeSBkZW3DoXMgbWVkaW9zIHF1ZSBzZSBlc3RpbWVuIG5lY2VzYXJpb3MgcGFyYSBwcm9tb3ZlciBlbCBhY2Nlc28geSBjb25zdWx0YSBkZSBsYSBtaXNtYS4gCgpnKQlFbiBlbCBjYXNvIGRlIGxhcyB0ZXNpcyBjcmVhZGFzIHBhcmEgb3B0YXIgZG9ibGUgdGl0dWxhY2nDs24sIGxvcyBmaXJtYW50ZXMgc2Vyw6FuIGxvcyByZXNwb25zYWJsZXMgZGUgY29tdW5pY2FyIGEgbGFzIGluc3RpdHVjaW9uZXMgbmFjaW9uYWxlcyBvIGV4dHJhbmplcmFzIGVuIGNvbnZlbmlvLCBsYXMgbGljZW5jaWFzIGRlIGFjY2VzbyBhYmllcnRvIENyZWF0aXZlIENvbW1vbnMgeSBhdXRvcml6YWNpb25lcyBhc2lnbmFkYXMgYSBzdSBvYnJhIHBhcmEgbGEgcHVibGljYWNpw7NuIGVuIGVsIFJlcG9zaXRvcmlvIEluc3RpdHVjaW9uYWwgVU5BTCBkZSBhY3VlcmRvIGNvbiBsYXMgZGlyZWN0cmljZXMgZGUgbGEgUG9sw610aWNhIEdlbmVyYWwgZGUgbGEgQmlibGlvdGVjYSBEaWdpdGFsLgoKCmgpCVNlIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgY29tbyByZXNwb25zYWJsZSBkZWwgdHJhdGFtaWVudG8gZGUgZGF0b3MgcGVyc29uYWxlcywgZGUgYWN1ZXJkbyBjb24gbGEgbGV5IDE1ODEgZGUgMjAxMiBlbnRlbmRpZW5kbyBxdWUgc2UgZW5jdWVudHJhbiBiYWpvIG1lZGlkYXMgcXVlIGdhcmFudGl6YW4gbGEgc2VndXJpZGFkLCBjb25maWRlbmNpYWxpZGFkIGUgaW50ZWdyaWRhZCwgeSBzdSB0cmF0YW1pZW50byB0aWVuZSB1bmEgZmluYWxpZGFkIGhpc3TDs3JpY2EsIGVzdGFkw61zdGljYSBvIGNpZW50w61maWNhIHNlZ8O6biBsbyBkaXNwdWVzdG8gZW4gbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMuCgoKClBBUlRFIDIuIEFVVE9SSVpBQ0nDk04gUEFSQSBQVUJMSUNBUiBZIFBFUk1JVElSIExBIENPTlNVTFRBIFkgVVNPIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KClNlIGF1dG9yaXphIGxhIHB1YmxpY2FjacOzbiBlbGVjdHLDs25pY2EsIGNvbnN1bHRhIHkgdXNvIGRlIGxhIG9icmEgcG9yIHBhcnRlIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgZGUgc3VzIHVzdWFyaW9zIGRlIGxhIHNpZ3VpZW50ZSBtYW5lcmE6CgphLglDb25jZWRvIGxpY2VuY2lhIGVuIGxvcyB0w6lybWlub3Mgc2XDsWFsYWRvcyBlbiBsYSBwYXJ0ZSAxIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8sIGNvbiBlbCBvYmpldGl2byBkZSBxdWUgbGEgb2JyYSBlbnRyZWdhZGEgc2VhIHB1YmxpY2FkYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgcHVlc3RhIGEgZGlzcG9zaWNpw7NuIGVuIGFjY2VzbyBhYmllcnRvIHBhcmEgc3UgY29uc3VsdGEgcG9yIGxvcyB1c3VhcmlvcyBkZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSAgYSB0cmF2w6lzIGRlIGludGVybmV0LgoKCgpQQVJURSAzIEFVVE9SSVpBQ0nDk04gREUgVFJBVEFNSUVOVE8gREUgREFUT1MgUEVSU09OQUxFUy4KCkxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLCBjb21vIHJlc3BvbnNhYmxlIGRlbCBUcmF0YW1pZW50byBkZSBEYXRvcyBQZXJzb25hbGVzLCBpbmZvcm1hIHF1ZSBsb3MgZGF0b3MgZGUgY2Fyw6FjdGVyIHBlcnNvbmFsIHJlY29sZWN0YWRvcyBtZWRpYW50ZSBlc3RlIGZvcm11bGFyaW8sIHNlIGVuY3VlbnRyYW4gYmFqbyBtZWRpZGFzIHF1ZSBnYXJhbnRpemFuIGxhIHNlZ3VyaWRhZCwgY29uZmlkZW5jaWFsaWRhZCBlIGludGVncmlkYWQgeSBzdSB0cmF0YW1pZW50byBzZSByZWFsaXphIGRlIGFjdWVyZG8gYWwgY3VtcGxpbWllbnRvIG5vcm1hdGl2byBkZSBsYSBMZXkgMTU4MSBkZSAyMDEyIHkgZGUgbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMgZGUgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEuIFB1ZWRlIGVqZXJjZXIgc3VzIGRlcmVjaG9zIGNvbW8gdGl0dWxhciBhIGNvbm9jZXIsIGFjdHVhbGl6YXIsIHJlY3RpZmljYXIgeSByZXZvY2FyIGxhcyBhdXRvcml6YWNpb25lcyBkYWRhcyBhIGxhcyBmaW5hbGlkYWRlcyBhcGxpY2FibGVzIGEgdHJhdsOpcyBkZSBsb3MgY2FuYWxlcyBkaXNwdWVzdG9zIHkgZGlzcG9uaWJsZXMgZW4gd3d3LnVuYWwuZWR1LmNvIG8gZS1tYWlsOiBwcm90ZWNkYXRvc19uYUB1bmFsLmVkdS5jbyIKClRlbmllbmRvIGVuIGN1ZW50YSBsbyBhbnRlcmlvciwgYXV0b3Jpem8gZGUgbWFuZXJhIHZvbHVudGFyaWEsIHByZXZpYSwgZXhwbMOtY2l0YSwgaW5mb3JtYWRhIGUgaW5lcXXDrXZvY2EgYSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhIHRyYXRhciBsb3MgZGF0b3MgcGVyc29uYWxlcyBkZSBhY3VlcmRvIGNvbiBsYXMgZmluYWxpZGFkZXMgZXNwZWPDrWZpY2FzIHBhcmEgZWwgZGVzYXJyb2xsbyB5IGVqZXJjaWNpbyBkZSBsYXMgZnVuY2lvbmVzIG1pc2lvbmFsZXMgZGUgZG9jZW5jaWEsIGludmVzdGlnYWNpw7NuIHkgZXh0ZW5zacOzbiwgYXPDrSBjb21vIGxhcyByZWxhY2lvbmVzIGFjYWTDqW1pY2FzLCBsYWJvcmFsZXMsIGNvbnRyYWN0dWFsZXMgeSB0b2RhcyBsYXMgZGVtw6FzIHJlbGFjaW9uYWRhcyBjb24gZWwgb2JqZXRvIHNvY2lhbCBkZSBsYSBVbml2ZXJzaWRhZC4gCgo=