Un método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learning

Ilustraciones, gráficos, tablas

Autores:
Saavedra Porras, Edher Daniel
Tipo de recurso:
Fecha de publicación:
2023
Institución:
Universidad Nacional de Colombia
Repositorio:
Universidad Nacional de Colombia
Idioma:
OAI Identifier:
oai:repositorio.unal.edu.co:unal/85793
Acceso en línea:
https://repositorio.unal.edu.co/handle/unal/85793
https://repositorio.unal.edu.co/
Palabra clave:
330 - Economía::332 - Economía financiera
Crédito
Sistemas de crédito - Colombia
Análisis financiero
Riesgo crediticio
Indicadores financieros
Sector financiero colombiano
CAMEL
Machine learning
Random forest
Credit risk
Financial indicators
Rights
openAccess
License
Atribución-NoComercial-SinDerivadas 4.0 Internacional
id UNACIONAL2_6e9ff12ad8195f7bd718e4a3e9da94db
oai_identifier_str oai:repositorio.unal.edu.co:unal/85793
network_acronym_str UNACIONAL2
network_name_str Universidad Nacional de Colombia
repository_id_str
dc.title.spa.fl_str_mv Un método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learning
dc.title.translated.eng.fl_str_mv A method for assigning credit quotas for entities in the Colombian financial sector using machine learning techniques
title Un método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learning
spellingShingle Un método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learning
330 - Economía::332 - Economía financiera
Crédito
Sistemas de crédito - Colombia
Análisis financiero
Riesgo crediticio
Indicadores financieros
Sector financiero colombiano
CAMEL
Machine learning
Random forest
Credit risk
Financial indicators
title_short Un método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learning
title_full Un método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learning
title_fullStr Un método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learning
title_full_unstemmed Un método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learning
title_sort Un método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learning
dc.creator.fl_str_mv Saavedra Porras, Edher Daniel
dc.contributor.advisor.none.fl_str_mv Espinosa Bedoya, Albeiro
dc.contributor.author.none.fl_str_mv Saavedra Porras, Edher Daniel
dc.subject.ddc.spa.fl_str_mv 330 - Economía::332 - Economía financiera
topic 330 - Economía::332 - Economía financiera
Crédito
Sistemas de crédito - Colombia
Análisis financiero
Riesgo crediticio
Indicadores financieros
Sector financiero colombiano
CAMEL
Machine learning
Random forest
Credit risk
Financial indicators
dc.subject.lemb.none.fl_str_mv Crédito
Sistemas de crédito - Colombia
Análisis financiero
dc.subject.proposal.spa.fl_str_mv Riesgo crediticio
Indicadores financieros
Sector financiero colombiano
dc.subject.proposal.eng.fl_str_mv CAMEL
Machine learning
Random forest
Credit risk
Financial indicators
description Ilustraciones, gráficos, tablas
publishDate 2023
dc.date.issued.none.fl_str_mv 2023-12-30
dc.date.accessioned.none.fl_str_mv 2024-03-11T15:41:09Z
dc.date.available.none.fl_str_mv 2024-03-11T15:41:09Z
dc.type.spa.fl_str_mv Trabajo de grado - Maestría
dc.type.driver.spa.fl_str_mv info:eu-repo/semantics/masterThesis
dc.type.version.spa.fl_str_mv info:eu-repo/semantics/acceptedVersion
dc.type.content.spa.fl_str_mv Text
dc.type.redcol.spa.fl_str_mv http://purl.org/redcol/resource_type/TM
status_str acceptedVersion
dc.identifier.uri.none.fl_str_mv https://repositorio.unal.edu.co/handle/unal/85793
dc.identifier.instname.spa.fl_str_mv Universidad Nacional de Colombia
dc.identifier.reponame.spa.fl_str_mv Repositorio Institucional Universidad Nacional de Colombia
dc.identifier.repourl.spa.fl_str_mv https://repositorio.unal.edu.co/
url https://repositorio.unal.edu.co/handle/unal/85793
https://repositorio.unal.edu.co/
identifier_str_mv Universidad Nacional de Colombia
Repositorio Institucional Universidad Nacional de Colombia
dc.relation.indexed.spa.fl_str_mv LaReferencia
dc.relation.references.spa.fl_str_mv [1] BRC Standard & Poor´s, “Metodología de calificación para instituciones financieras Emisores de deuda”, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.brc.com.co/archivos/3_Tipos_Metodologias_calificacion/3_3_Metodologias_calificaciones/3_3_1_sector_financiero/3_3_1_1_Establec_credito/cal-met-005%20Met%20InsFinanDeuda%20V3.pdf.
[2] J.Y. Crespo, “CAMEL vs. discriminante, un análisis de riesgo al sistema financiero venezolano” Scielo, vol.15 no.33, Dec. 2011, http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S1657-42062011000200002.
[3] W. Ossa y V. Jaramillo, “Machine Learning para la estimación del riesgo de crédito en una cartera de consumo”, Universidad EAFIT, 2021, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://repository.eafit.edu.co/bitstream/handle/10784/29589/Wbeimar_OssaGiraldo_Veronica_JaramilloMarin_2021.pdf?sequence=2&isAllowed=y.
[4] J. Grau, “Machine learning y riesgo de crédito”, Universidad Pontificia Comillas, 2020, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://repositorio.comillas.edu/xmlui/bitstream/handle/11531/39062/TFG-%20Grau%20Alvarez%2C%20Jaime.pdf?sequence=1&isAllowed=y.
[5] D. Borrero y O. Bedoya, “Predicción de riesgo crediticio en Colombia usando técnicas de inteligencia artificial”, UIS Ingenierías, vol. 19, Universidad Industrial de Santander, jul. 2020, doi: https://doi.org/10.18273/revuin.v19n4-2020004.
[6] Superintendencia Financiera de Colombia, “Capítulo II: Reglas relativas a la gestión de riesgo crediticio”, Circular Externa 010, 2008, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://fasecolda.com/cms/wp-content/uploads/2019/08/ce100-1995-cap-ii.pdf.
[7] Cámara de Comercio de Oviedo, “Solvencia Financiera”, 2021, https://www.mba-asturias.com/economia/que-es-solvencia-finanzas/.
[8] F. D. Freitas, A. F. de Souza, and A. R. de Almeida, “Prediction-based portfolio optimization model using neural networks,” Neurocomputing, vol. 72, no. 10–12, pp. 2155–2170, Jun. 2009, doi: 10.1016/j.neucom.2008.08.019.
[9] IBM, “¿Qué es Machine Learning?”, https://www.ibm.com/co-es/analytics/machine-learning.
[10] K. Tamás & V. Miklós, “EU-27 bank failure prediction with C5.0 decision trees and deep learning neural networks”, Research in International Business and Finance, vol. 61, 2022, doi: 10.1016/j.ribaf.2022.101644.
[11] Z. Yao & G. Changchun, “Extreme Learning Machine Enhanced Gradient Boosting for Credit Scoring”, Algorithms, vol. 15, 2022, doi: 10.3390/a15050149.
[12] A. Samar & A. Marco, “Developing An Intelligent System For Predicting Bankruptcy”, Journal of Theoretical and Applied Information Technology, vol. 100, 2022. https://www.scopus.com/record/display.uri?eid=2-s2.0-85128612727&origin=resultslist&sort=plf-f&src=s&st1=credit+risk&st2=machine+learning&sid=f9bb9a8ef9189a277ef72e4bfedc0578&sot=b&sdt=b&sl=64&s=%28TITLE-ABS-KEY%28credit+risk%29+AND+TITLE-ABS-KEY%28machine+learning%29%29&relpos=10&citeCnt=0&searchTerm=&featureToggles=FEATURE_NEW_DOC_DETAILS_EXPORT:1.
[13] A. Srinivas & R. Somula, “Loan Default Prediction Using Machine Learning Techniques”, Lecture Notes in Networks and Systems, vol. 385, 2022, doi: 10.1007/978-981-16-8987-1_56.
[14] T. Germanno, J. Rodrigues, R. Ricardo y K. Sergei, “Comparative study of support vector machines and random forests machine learning algorithms on credit operation”, Software - Practice and Experience, vol. 51, 2021, doi: 10.1002/spe.2842.
[15] L. Yun, X. Zhang y H. Yin, “An extreme learning machine based virtual sample generation method with feature engineering for credit risk assessment with data scarcity”, Expert Systems with Applications, vol.202, 2022, doi: 10.1016/j.eswa.2022.117363.
[16] W. Guo y Z. Zhou, “A comparative study of combining tree-based feature selection methods and classifiers in personal loan default prediction”, Journal of Forecasting, vol. 41, 2022, doi: 10.1002/for.2856.
[17] R. García y M. Moreno, “The generalized Vasicek credit risk model: A Machine Learning approach”, Finance Research Letters, vol. 47, 2022, doi: 10.1016/j.frl.2021.102669.
[18] T. Wang, R. Liu y G. Qi, “Multi-classification assessment of bank personal credit risk based on multi-source information fusion”, Expert Systems with Applications, vol. 191, 2022, doi: 10.1016/j.eswa.2021.116236.
[19] E. Dumitrescu, S. Hué, C. Hurlin y S. Tokpavi, “Machine learning for credit scoring: Improving logistic regression with non-linear decision-tree effects”, European Journal of Operational Research, vol. 297, 2022, doi: 10.1016/j.ejor.2021.06.053.
[20] S. Dogan, Y. Buyukkor y M. Atan, “A COMPARATIVE STUDY OF CORPORATE CREDIT RATING PREDICTION WITH MACHINE LEARNING”, Operations Research and Decisions, vol. 32, 2022, doi: 10.37190/ord220102.
[21] Wang, C., Liu, Q., Li, S., “A two-stage credit risk scoring method with stacked-generalisation ensemble learning in peer-to-peer lending”, International Journal of Embedded Systems, vol. 15, 2022, doi: 10.1504/IJES.2022.123312.
[22] Jadwal, P.K., Pathak, S., Jain, S., “Analysis of clustering algorithms for credit risk evaluation using multiple correspondence analysis”, Microsystem Technologies, 2022, doi: 10.1007/s00542-022-05310-y.
[23] Tian, J., Li, L., “Digital Universal Financial Credit Risk Analysis Using Particle Swarm Optimization Algorithm with Structure Decision Tree Learning-Based Evaluation Model”, Wireless Communications and Mobile Computing, vol. 2022, 2022, doi: 10.1155/2022/4060256.
[24] Coenen, L., Verbeke, W., Guns, T., “Machine learning methods for short-term probability of default: A comparison of classification, regression and ranking methods”, Journal of the Operational Research Society, vol. 73, 2022, doi: 10.1080/01605682.2020.1865847.
[25] Frydman, H., Matuszyk, A., “Random survival forest for competing credit risks”, Journal of the Operational Research Society, vol. 73, 2022, doi: 10.1080/01605682.2020.1759385.
[26] Hu, L., Chen, J., Vaughan, J., (...), Sudjianto, A., Nair, V.N., “Supervised Machine Learning Techniques: An Overview with Applications to Banking”, International Statistical Review, vol. 89, 2021, doi: 10.1111/insr.12448.
[27] Gao, G., Wang, H., Gao, P., “Establishing a credit risk evaluation system for smes using the soft voting fusion model”, Risks, vol. 9, 2021, doi: 10.3390/risks9110202.
[28] Wu, C.-F., Huang, S.-C., Chiou, C.-C., Wang, Y.-M., “A predictive intelligence system of credit scoring based on deep multiple kernel learning”, Applied Soft Computing, vol. 111, 2021, doi: 10.1016/j.asoc.2021.107668.
[29] Acheampong, A., Elshandidy, T., “Does soft information determine credit risk? Text-based evidence from European Banks”, Journal of International Financial Markets, Institutions and Money, vol. 75, 2021, doi: 10.1016/j.intfin.2021.101303.
[30] Kokate, S., Chetty, M.S.R., “Credit risk assessment of loan defaulters in commercial banks using voting classifier ensemble learner machine learning model”, International Journal of Safety and Security Engineering, vol. 11, 2021, doi: 10.18280/IJSSE.110508.
[31] Breeden, J.L., “A survey of machine learning in credit risk”, Journal of Credit Risk, vol. 17, 2021, doi: 10.21314/JCR.2021.008.
[32] Liu, R., Yang, X., Dong, X., Sun, B., “Credit risk assessment of banks' loan enterprise customer based on state-constraint”, Computing and Informatics, vol. 40, 2021, doi: 10.31577/cai_2021_1_145.
[33] Lappas, P.Z., Yannacopoulos, A.N., “A machine learning approach combining expert knowledge with genetic algorithms in feature selection for credit risk assessment”, Applied Soft Computing, vol. 107, 2021, doi: 10.1016/j.asoc.2021.107391.
[34] Balakrishnan, C., Thiagarajan, M., “Credit risk modelling for Indian debt securities using machine learning”, Buletin Ekonomi Moneter dan Perbankan, vol. 24, 2021, doi: 10.21098/BEMP.V24I0.1401.
[35] Ampountolas, A., Nde, T.N., Date, P., Constantinescu, C., “A machine learning approach for micro-credit scoring”, Risks, vol. 9, 2021, doi: 10.3390/risks9030050.
[36] Moscato, V., Picariello, A., Sperlí, G., “A benchmark of machine learning approaches for credit score prediction”, Expert Systems with Applications, vol. 165, 2021, doi: 10.1016/j.eswa.2020.113986.
[37] Shen, F., Zhao, X., Kou, G., & Alsaadi, F. E., “A new deep learning ensemble credit risk evaluation model with an improved synthetic minority oversampling technique”, Applied Soft Computing, vol. 98, 2021, 106852.
[38] Moscatelli, M., Parlapiano, F., Narizzano, S., & Viggiano, G., “Corporate default forecasting with machine learning”, Expert Systems with Applications, vol. 161, 2020, 113567.
[39] Li, J. P., Mirza, N., Rahat, B., & Xiong, D., “Machine learning and credit ratings prediction in the age of fourth industrial revolution”, Technological Forecasting and Social Change, vol. 161, 2020, 120309.
[40] Tripathi, D., Edla, D. R., Kuppili, V., & Bablani, A., “Evolutionary extreme learning machine with novel activation function for credit scoring”, Engineering Applications of Artificial Intelligence, vol. 96, 2020, 103980.
[41] Luo, C., “A comprehensive decision support approach for credit scoring”, Industrial Management & Data Systems”, 2019, Vol. 120 No. 2, pp. 280-290. https://doi.org/10.1108/IMDS-03-2019-0182.
[42] Arora, N., & Kaur, P. D., “A Bolasso based consistent feature selection enabled random forest classification algorithm: An application to credit risk assessment”, Applied Soft Computing, vol. 86, 2020, 105936.
[43] Qasem, M. H., & Nemer, L., “Extreme Learning Machine for Credit Risk Analysis”, Journal of Intelligent Systems, vol. 29(1), 2020, 640-652.
[44] Superintendencia Financiera de Colombia, “Actualidad del Sistema Financiero”, 2022. https://www.superfinanciera.gov.co/inicio/informes-y-cifras/informes/informe-actualidad-del-sistema-financiero-colombiano/resultados-del-sistema-financiero-colombiano-agosto-de--10112528.
[45] Escobar-Pérez, J. y Cuervo-Martínez, A., “Validez de contenido y juicio de expertos: una aproximación a su utilización”, Avances en Medición, vol. 6, pp. 27-36, 2008, http://www.humanas.unal.edu.co/psicometria/files/7113/8574/5708/Articulo3_Juicio_de_expertos_27-36.pdf.
[46] Cabero Almenara, J. y Llorente Cejudo, M. C., “La aplicación del juicio de experto como técnica de evaluación de las tecnologías de la información (TIC)”. Eduweb. Revista de Tecnología de Información y Comunicación en Educación, vol. 7 (2) pp.11-22, 2013, http://tecnologiaedu.us.es/tecnoedu/images/stories/jca107.pdf
[47] Martínez, J. (2020). Precision, Recall, F1, Accuracy en clasificación. IArtificial.net. https://www.iartificial.net/precision-recall-f1-accuracy-en-clasificacion/
[48] Sanahuja, P. (2021). Entendiendo la curva ROC y el AUC: Dos medidas del rendimiento de un clasificador binario que van de la mano. Pol Martí Sanahuja. https://polmartisanahuja.com/entendiendo-la-curva-roc-y-el-auc-dos-medidas-del-rendimiento-de-un-clasificador-binario-que-van-de-la-mano/.
[49] Amat, J. (2016). Regresión logística simple y múltiple. Cienciadedatos.net. https://www.cienciadedatos.net/documentos/27_regresion_logistica_simple_y_multiple.
[50] Cuenca, D. & León, D. (2022). Support Vector Machine. Amazon AWS. https://rstudio-pubs-static.s3.amazonaws.com/570352_e34015b16f1a47e883e04c6195d4711f.html.
[51] Amat, J. (2017). Máquinas de Vector Soporte (Support Vector Machines, SVMs). Cienciadedatos.net. https://www.cienciadedatos.net/documentos/34_maquinas_de_vector_soporte_support_vector_machines.
[52] Amazon AWS (2023). ¿Qué es una red neuronal?. Amazon AWS. https://aws.amazon.com/es/what-is/neural-network/.
[53] Matich, D. (2001). Redes Neuronales: Conceptos Básicos y Aplicaciones. Universidad Tecnológica Nacional. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.frro.utn.edu.ar/repositorio/catedras/quimica/5_anio/orientadora1/monograias/matich-redesneuronales.pdf.
[54] Aprendemachinelearning.com (2018). Clasificar con K-Nearest-Neighbor ejemplo en Python. Aprendemachinelearning.com. https://www.aprendemachinelearning.com/clasificar-con-k-nearest-neighbor-ejemplo-en-python/.
[55] Ferrero, R. (2020). Qué son los árboles de decisión y para qué sirven. Máximaformacion.es. https://www.maximaformacion.es/blog-dat/que-son-los-arboles-de-decision-y-para-que-sirven/.
[56] Martínez, J. (2020). Random Forest (Bosque Aleatorio): combinando árboles. Iartificial.net. https://www.iartificial.net/random-forest-bosque-aleatorio/.
[57] IBM (2023). ¿Qué es un bosque aleatorio? IBM. https://www.ibm.com/mx-es/topics/random-forest.
[58] Unidad de Regulación Financiera (2023). Decreto Único 2555 de 2010. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.urf.gov.co/webcenter/ShowProperty?nodeId=/ConexionContent/WCC_CLUSTER-107284.
[59] Asobancaria (2019). Metodología de selección de las entidades financieras que participarán en el esquema del indicador bancario de referencia. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.asobancaria.com/wp-content/uploads/2019-08-metodologia-de-seleccion-entidades-actualizada-VF.pdf.
[60] ARL SURA (2023). Tomado del artículo publicado en la revista Gerencia de Riesgos y Seguros de la Fundación MAPFRE ESTUDIOS. https://www.arlsura.com/index.php/66-centro-de-documentacion-anterior/prevencion-de-riesgos-/280--sp-28739.
[61] Amazon Web Services (2023). ¿Qué es el sobreajuste?. https://aws.amazon.com/es/what-is/overfitting/.
dc.rights.coar.fl_str_mv http://purl.org/coar/access_right/c_abf2
dc.rights.license.spa.fl_str_mv Atribución-NoComercial-SinDerivadas 4.0 Internacional
dc.rights.uri.spa.fl_str_mv http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.accessrights.spa.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv Atribución-NoComercial-SinDerivadas 4.0 Internacional
http://creativecommons.org/licenses/by-nc-nd/4.0/
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.extent.spa.fl_str_mv 57 páginas
dc.format.mimetype.spa.fl_str_mv application/pdf
dc.publisher.spa.fl_str_mv Universidad Nacional de Colombia
dc.publisher.program.spa.fl_str_mv Medellín - Minas - Maestría en Ingeniería - Analítica
dc.publisher.faculty.spa.fl_str_mv Facultad de Minas
dc.publisher.place.spa.fl_str_mv Medellín, Colombia
dc.publisher.branch.spa.fl_str_mv Universidad Nacional de Colombia - Sede Medellín
institution Universidad Nacional de Colombia
bitstream.url.fl_str_mv https://repositorio.unal.edu.co/bitstream/unal/85793/1/license.txt
https://repositorio.unal.edu.co/bitstream/unal/85793/2/1128400916.2024.pdf
https://repositorio.unal.edu.co/bitstream/unal/85793/3/1128400916.2024.pdf.jpg
bitstream.checksum.fl_str_mv eb34b1cf90b7e1103fc9dfd26be24b4a
b8f3f112e7365a4a3d6a657f8fa32bef
21db169f4c0ca26c0e9beba3fbb09482
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
MD5
repository.name.fl_str_mv Repositorio Institucional Universidad Nacional de Colombia
repository.mail.fl_str_mv repositorio_nal@unal.edu.co
_version_ 1814089467291500544
spelling Atribución-NoComercial-SinDerivadas 4.0 Internacionalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccesshttp://purl.org/coar/access_right/c_abf2Espinosa Bedoya, Albeiro749aa8775c497b18160b8a0a5d502335Saavedra Porras, Edher Daniel021f4e86694cd81017d6e9fae00a493b2024-03-11T15:41:09Z2024-03-11T15:41:09Z2023-12-30https://repositorio.unal.edu.co/handle/unal/85793Universidad Nacional de ColombiaRepositorio Institucional Universidad Nacional de Colombiahttps://repositorio.unal.edu.co/Ilustraciones, gráficos, tablasEl análisis de riesgo de crédito desempeña un papel crucial en el sector financiero, ya que evalúa variables que podrían deteriorarse en circunstancias particulares, llevando a un incumplimiento de obligaciones y, aunque la metodología CAMEL es ampliamente utilizada, esta se basa en un sistema de calificaciones simple. Evaluar la influencia de factores macro y microeconómicos en la estabilidad financiera, especialmente en un contexto de preocupaciones globales, junto con la monitorización de indicadores es crucial para mitigar incumplimientos crediticios. A agosto de 2022, la cartera morosa en el sector financiero colombiano asciende a COP 23.4 billones, por lo cual, se busca proponer un método para la asignación de cupos de crédito basado en machine learning, analizar métodos existentes, desarrollar un método basado en indicadores financieros, y validar el modelo propuesto comparándolo con otros en la literatura. El análisis de los antecedentes muestra que los métodos de machine learning superan a los estadísticos tradicionales en la estimación de riesgos crediticios, destacándose técnicas como Random Forest, Support Vector Machines, y redes neuronales. Además, se aplicaron criterios ponderados para evaluar la elección de dichos métodos, considerando la frecuencia de aplicación, resultados destacados en la literatura y opiniones de expertos. Random Forest y Árboles de Decisión obtuvieron las calificaciones más altas en el ranking debido a que se destacan su flexibilidad y capacidad para manejar diversos desafíos en diferentes aplicaciones. El análisis se basó en más de 50 indicadores financieros recopilados de la Superintendencia Financiera de Colombia, abarcando diversas entidades del sector financiero, para luego implementar un modelo de clasificación de riesgo crediticio mediante Random Forest, logrando una precisión excepcional del 99.9% en datos de prueba y 99.79% en entrenamiento. La interpretación de la importancia de las características y la matriz de confusión respaldan la robustez del modelo. Finalmente, se compararon los resultados con árboles de decisión y regresión logística, obteniendo un accuracy de 99.9% para Random Forest, 97.9% en la métrica de recall y 98.9% en F1 Score, resultados superiores a los modelos en comparación y destacando el notable rendimiento superior de Random Forest en la predicción de riesgo crediticio. Estos hallazgos respaldan su elección como una herramienta eficaz en la gestión de riesgos crediticios en el contexto colombiano. (Tomado de la fuente)Credit risk analysis plays a crucial role in the financial sector, as it evaluates variables that could deteriorate under specific circumstances, leading to non-compliance with obligations. Although the CAMEL methodology is widely used, it relies on a simple rating system. Evaluating the influence of macro and microeconomic factors on financial stability, especially in a context of global concerns, and monitoring indicators are crucial to mitigate credit defaults. As of August 2022, the non-performing portfolio in the Colombian financial sector amounts to COP 23.4 trillion. Therefore, we aim to propose a method for assigning credit quotas based on machine learning, analyze existing methods, develop a method based on financial indicators, and validate the proposed model by comparing it with others in the literature. The background analysis indicates that machine learning methods outperform traditional statistics in estimating credit risks. Techniques such as Random Forest, Support Vector Machines, and neural networks stand out. Weighted criteria were applied to evaluate the choice of these methods, considering the frequency of application, notable results in the literature, and expert opinions. Random Forest and Decision Trees obtained the highest scores in the ranking due to their flexibility and ability to handle various challenges in different applications. The analysis was based on more than 50 financial indicators collected from the Financial Superintendency of Colombia, encompassing various entities in the financial sector. Subsequently, a credit risk classification model was implemented using Random Forest, achieving an exceptional accuracy of 99.9% in test data and 99.79% in training. The interpretation of the importance of the features and the confusion matrix supports the robustness of the model. Finally, the results were compared with decision trees and logistic regression, yielding an accuracy of 99.9% for Random Forest, 97.9% in the recall metric, and 98.9% in F1 Score, results superior to the models in comparison, highlighting the remarkable superior performance of Random Forest in credit risk prediction. These findings support its selection as an effective tool in credit risk management in the Colombian context.MaestríaMagíster en Ingeniería - AnalíticaIngeniería De Sistemas E Informática.Sede Medellín57 páginasapplication/pdfUniversidad Nacional de ColombiaMedellín - Minas - Maestría en Ingeniería - AnalíticaFacultad de MinasMedellín, ColombiaUniversidad Nacional de Colombia - Sede Medellín330 - Economía::332 - Economía financieraCréditoSistemas de crédito - ColombiaAnálisis financieroRiesgo crediticioIndicadores financierosSector financiero colombianoCAMELMachine learningRandom forestCredit riskFinancial indicatorsUn método para la asignación de cupos de crédito de entidades del sector financiero colombiano empleando técnicas de machine learningA method for assigning credit quotas for entities in the Colombian financial sector using machine learning techniquesTrabajo de grado - Maestríainfo:eu-repo/semantics/masterThesisinfo:eu-repo/semantics/acceptedVersionTexthttp://purl.org/redcol/resource_type/TMLaReferencia[1] BRC Standard & Poor´s, “Metodología de calificación para instituciones financieras Emisores de deuda”, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.brc.com.co/archivos/3_Tipos_Metodologias_calificacion/3_3_Metodologias_calificaciones/3_3_1_sector_financiero/3_3_1_1_Establec_credito/cal-met-005%20Met%20InsFinanDeuda%20V3.pdf.[2] J.Y. Crespo, “CAMEL vs. discriminante, un análisis de riesgo al sistema financiero venezolano” Scielo, vol.15 no.33, Dec. 2011, http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S1657-42062011000200002.[3] W. Ossa y V. Jaramillo, “Machine Learning para la estimación del riesgo de crédito en una cartera de consumo”, Universidad EAFIT, 2021, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://repository.eafit.edu.co/bitstream/handle/10784/29589/Wbeimar_OssaGiraldo_Veronica_JaramilloMarin_2021.pdf?sequence=2&isAllowed=y.[4] J. Grau, “Machine learning y riesgo de crédito”, Universidad Pontificia Comillas, 2020, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://repositorio.comillas.edu/xmlui/bitstream/handle/11531/39062/TFG-%20Grau%20Alvarez%2C%20Jaime.pdf?sequence=1&isAllowed=y.[5] D. Borrero y O. Bedoya, “Predicción de riesgo crediticio en Colombia usando técnicas de inteligencia artificial”, UIS Ingenierías, vol. 19, Universidad Industrial de Santander, jul. 2020, doi: https://doi.org/10.18273/revuin.v19n4-2020004.[6] Superintendencia Financiera de Colombia, “Capítulo II: Reglas relativas a la gestión de riesgo crediticio”, Circular Externa 010, 2008, chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://fasecolda.com/cms/wp-content/uploads/2019/08/ce100-1995-cap-ii.pdf.[7] Cámara de Comercio de Oviedo, “Solvencia Financiera”, 2021, https://www.mba-asturias.com/economia/que-es-solvencia-finanzas/.[8] F. D. Freitas, A. F. de Souza, and A. R. de Almeida, “Prediction-based portfolio optimization model using neural networks,” Neurocomputing, vol. 72, no. 10–12, pp. 2155–2170, Jun. 2009, doi: 10.1016/j.neucom.2008.08.019.[9] IBM, “¿Qué es Machine Learning?”, https://www.ibm.com/co-es/analytics/machine-learning.[10] K. Tamás & V. Miklós, “EU-27 bank failure prediction with C5.0 decision trees and deep learning neural networks”, Research in International Business and Finance, vol. 61, 2022, doi: 10.1016/j.ribaf.2022.101644.[11] Z. Yao & G. Changchun, “Extreme Learning Machine Enhanced Gradient Boosting for Credit Scoring”, Algorithms, vol. 15, 2022, doi: 10.3390/a15050149.[12] A. Samar & A. Marco, “Developing An Intelligent System For Predicting Bankruptcy”, Journal of Theoretical and Applied Information Technology, vol. 100, 2022. https://www.scopus.com/record/display.uri?eid=2-s2.0-85128612727&origin=resultslist&sort=plf-f&src=s&st1=credit+risk&st2=machine+learning&sid=f9bb9a8ef9189a277ef72e4bfedc0578&sot=b&sdt=b&sl=64&s=%28TITLE-ABS-KEY%28credit+risk%29+AND+TITLE-ABS-KEY%28machine+learning%29%29&relpos=10&citeCnt=0&searchTerm=&featureToggles=FEATURE_NEW_DOC_DETAILS_EXPORT:1.[13] A. Srinivas & R. Somula, “Loan Default Prediction Using Machine Learning Techniques”, Lecture Notes in Networks and Systems, vol. 385, 2022, doi: 10.1007/978-981-16-8987-1_56.[14] T. Germanno, J. Rodrigues, R. Ricardo y K. Sergei, “Comparative study of support vector machines and random forests machine learning algorithms on credit operation”, Software - Practice and Experience, vol. 51, 2021, doi: 10.1002/spe.2842.[15] L. Yun, X. Zhang y H. Yin, “An extreme learning machine based virtual sample generation method with feature engineering for credit risk assessment with data scarcity”, Expert Systems with Applications, vol.202, 2022, doi: 10.1016/j.eswa.2022.117363.[16] W. Guo y Z. Zhou, “A comparative study of combining tree-based feature selection methods and classifiers in personal loan default prediction”, Journal of Forecasting, vol. 41, 2022, doi: 10.1002/for.2856.[17] R. García y M. Moreno, “The generalized Vasicek credit risk model: A Machine Learning approach”, Finance Research Letters, vol. 47, 2022, doi: 10.1016/j.frl.2021.102669.[18] T. Wang, R. Liu y G. Qi, “Multi-classification assessment of bank personal credit risk based on multi-source information fusion”, Expert Systems with Applications, vol. 191, 2022, doi: 10.1016/j.eswa.2021.116236.[19] E. Dumitrescu, S. Hué, C. Hurlin y S. Tokpavi, “Machine learning for credit scoring: Improving logistic regression with non-linear decision-tree effects”, European Journal of Operational Research, vol. 297, 2022, doi: 10.1016/j.ejor.2021.06.053.[20] S. Dogan, Y. Buyukkor y M. Atan, “A COMPARATIVE STUDY OF CORPORATE CREDIT RATING PREDICTION WITH MACHINE LEARNING”, Operations Research and Decisions, vol. 32, 2022, doi: 10.37190/ord220102.[21] Wang, C., Liu, Q., Li, S., “A two-stage credit risk scoring method with stacked-generalisation ensemble learning in peer-to-peer lending”, International Journal of Embedded Systems, vol. 15, 2022, doi: 10.1504/IJES.2022.123312.[22] Jadwal, P.K., Pathak, S., Jain, S., “Analysis of clustering algorithms for credit risk evaluation using multiple correspondence analysis”, Microsystem Technologies, 2022, doi: 10.1007/s00542-022-05310-y.[23] Tian, J., Li, L., “Digital Universal Financial Credit Risk Analysis Using Particle Swarm Optimization Algorithm with Structure Decision Tree Learning-Based Evaluation Model”, Wireless Communications and Mobile Computing, vol. 2022, 2022, doi: 10.1155/2022/4060256.[24] Coenen, L., Verbeke, W., Guns, T., “Machine learning methods for short-term probability of default: A comparison of classification, regression and ranking methods”, Journal of the Operational Research Society, vol. 73, 2022, doi: 10.1080/01605682.2020.1865847.[25] Frydman, H., Matuszyk, A., “Random survival forest for competing credit risks”, Journal of the Operational Research Society, vol. 73, 2022, doi: 10.1080/01605682.2020.1759385.[26] Hu, L., Chen, J., Vaughan, J., (...), Sudjianto, A., Nair, V.N., “Supervised Machine Learning Techniques: An Overview with Applications to Banking”, International Statistical Review, vol. 89, 2021, doi: 10.1111/insr.12448.[27] Gao, G., Wang, H., Gao, P., “Establishing a credit risk evaluation system for smes using the soft voting fusion model”, Risks, vol. 9, 2021, doi: 10.3390/risks9110202.[28] Wu, C.-F., Huang, S.-C., Chiou, C.-C., Wang, Y.-M., “A predictive intelligence system of credit scoring based on deep multiple kernel learning”, Applied Soft Computing, vol. 111, 2021, doi: 10.1016/j.asoc.2021.107668.[29] Acheampong, A., Elshandidy, T., “Does soft information determine credit risk? Text-based evidence from European Banks”, Journal of International Financial Markets, Institutions and Money, vol. 75, 2021, doi: 10.1016/j.intfin.2021.101303.[30] Kokate, S., Chetty, M.S.R., “Credit risk assessment of loan defaulters in commercial banks using voting classifier ensemble learner machine learning model”, International Journal of Safety and Security Engineering, vol. 11, 2021, doi: 10.18280/IJSSE.110508.[31] Breeden, J.L., “A survey of machine learning in credit risk”, Journal of Credit Risk, vol. 17, 2021, doi: 10.21314/JCR.2021.008.[32] Liu, R., Yang, X., Dong, X., Sun, B., “Credit risk assessment of banks' loan enterprise customer based on state-constraint”, Computing and Informatics, vol. 40, 2021, doi: 10.31577/cai_2021_1_145.[33] Lappas, P.Z., Yannacopoulos, A.N., “A machine learning approach combining expert knowledge with genetic algorithms in feature selection for credit risk assessment”, Applied Soft Computing, vol. 107, 2021, doi: 10.1016/j.asoc.2021.107391.[34] Balakrishnan, C., Thiagarajan, M., “Credit risk modelling for Indian debt securities using machine learning”, Buletin Ekonomi Moneter dan Perbankan, vol. 24, 2021, doi: 10.21098/BEMP.V24I0.1401.[35] Ampountolas, A., Nde, T.N., Date, P., Constantinescu, C., “A machine learning approach for micro-credit scoring”, Risks, vol. 9, 2021, doi: 10.3390/risks9030050.[36] Moscato, V., Picariello, A., Sperlí, G., “A benchmark of machine learning approaches for credit score prediction”, Expert Systems with Applications, vol. 165, 2021, doi: 10.1016/j.eswa.2020.113986.[37] Shen, F., Zhao, X., Kou, G., & Alsaadi, F. E., “A new deep learning ensemble credit risk evaluation model with an improved synthetic minority oversampling technique”, Applied Soft Computing, vol. 98, 2021, 106852.[38] Moscatelli, M., Parlapiano, F., Narizzano, S., & Viggiano, G., “Corporate default forecasting with machine learning”, Expert Systems with Applications, vol. 161, 2020, 113567.[39] Li, J. P., Mirza, N., Rahat, B., & Xiong, D., “Machine learning and credit ratings prediction in the age of fourth industrial revolution”, Technological Forecasting and Social Change, vol. 161, 2020, 120309.[40] Tripathi, D., Edla, D. R., Kuppili, V., & Bablani, A., “Evolutionary extreme learning machine with novel activation function for credit scoring”, Engineering Applications of Artificial Intelligence, vol. 96, 2020, 103980.[41] Luo, C., “A comprehensive decision support approach for credit scoring”, Industrial Management & Data Systems”, 2019, Vol. 120 No. 2, pp. 280-290. https://doi.org/10.1108/IMDS-03-2019-0182.[42] Arora, N., & Kaur, P. D., “A Bolasso based consistent feature selection enabled random forest classification algorithm: An application to credit risk assessment”, Applied Soft Computing, vol. 86, 2020, 105936.[43] Qasem, M. H., & Nemer, L., “Extreme Learning Machine for Credit Risk Analysis”, Journal of Intelligent Systems, vol. 29(1), 2020, 640-652.[44] Superintendencia Financiera de Colombia, “Actualidad del Sistema Financiero”, 2022. https://www.superfinanciera.gov.co/inicio/informes-y-cifras/informes/informe-actualidad-del-sistema-financiero-colombiano/resultados-del-sistema-financiero-colombiano-agosto-de--10112528.[45] Escobar-Pérez, J. y Cuervo-Martínez, A., “Validez de contenido y juicio de expertos: una aproximación a su utilización”, Avances en Medición, vol. 6, pp. 27-36, 2008, http://www.humanas.unal.edu.co/psicometria/files/7113/8574/5708/Articulo3_Juicio_de_expertos_27-36.pdf.[46] Cabero Almenara, J. y Llorente Cejudo, M. C., “La aplicación del juicio de experto como técnica de evaluación de las tecnologías de la información (TIC)”. Eduweb. Revista de Tecnología de Información y Comunicación en Educación, vol. 7 (2) pp.11-22, 2013, http://tecnologiaedu.us.es/tecnoedu/images/stories/jca107.pdf[47] Martínez, J. (2020). Precision, Recall, F1, Accuracy en clasificación. IArtificial.net. https://www.iartificial.net/precision-recall-f1-accuracy-en-clasificacion/[48] Sanahuja, P. (2021). Entendiendo la curva ROC y el AUC: Dos medidas del rendimiento de un clasificador binario que van de la mano. Pol Martí Sanahuja. https://polmartisanahuja.com/entendiendo-la-curva-roc-y-el-auc-dos-medidas-del-rendimiento-de-un-clasificador-binario-que-van-de-la-mano/.[49] Amat, J. (2016). Regresión logística simple y múltiple. Cienciadedatos.net. https://www.cienciadedatos.net/documentos/27_regresion_logistica_simple_y_multiple.[50] Cuenca, D. & León, D. (2022). Support Vector Machine. Amazon AWS. https://rstudio-pubs-static.s3.amazonaws.com/570352_e34015b16f1a47e883e04c6195d4711f.html.[51] Amat, J. (2017). Máquinas de Vector Soporte (Support Vector Machines, SVMs). Cienciadedatos.net. https://www.cienciadedatos.net/documentos/34_maquinas_de_vector_soporte_support_vector_machines.[52] Amazon AWS (2023). ¿Qué es una red neuronal?. Amazon AWS. https://aws.amazon.com/es/what-is/neural-network/.[53] Matich, D. (2001). Redes Neuronales: Conceptos Básicos y Aplicaciones. Universidad Tecnológica Nacional. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.frro.utn.edu.ar/repositorio/catedras/quimica/5_anio/orientadora1/monograias/matich-redesneuronales.pdf.[54] Aprendemachinelearning.com (2018). Clasificar con K-Nearest-Neighbor ejemplo en Python. Aprendemachinelearning.com. https://www.aprendemachinelearning.com/clasificar-con-k-nearest-neighbor-ejemplo-en-python/.[55] Ferrero, R. (2020). Qué son los árboles de decisión y para qué sirven. Máximaformacion.es. https://www.maximaformacion.es/blog-dat/que-son-los-arboles-de-decision-y-para-que-sirven/.[56] Martínez, J. (2020). Random Forest (Bosque Aleatorio): combinando árboles. Iartificial.net. https://www.iartificial.net/random-forest-bosque-aleatorio/.[57] IBM (2023). ¿Qué es un bosque aleatorio? IBM. https://www.ibm.com/mx-es/topics/random-forest.[58] Unidad de Regulación Financiera (2023). Decreto Único 2555 de 2010. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.urf.gov.co/webcenter/ShowProperty?nodeId=/ConexionContent/WCC_CLUSTER-107284.[59] Asobancaria (2019). Metodología de selección de las entidades financieras que participarán en el esquema del indicador bancario de referencia. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.asobancaria.com/wp-content/uploads/2019-08-metodologia-de-seleccion-entidades-actualizada-VF.pdf.[60] ARL SURA (2023). Tomado del artículo publicado en la revista Gerencia de Riesgos y Seguros de la Fundación MAPFRE ESTUDIOS. https://www.arlsura.com/index.php/66-centro-de-documentacion-anterior/prevencion-de-riesgos-/280--sp-28739.[61] Amazon Web Services (2023). ¿Qué es el sobreajuste?. https://aws.amazon.com/es/what-is/overfitting/.Público generalLICENSElicense.txtlicense.txttext/plain; charset=utf-85879https://repositorio.unal.edu.co/bitstream/unal/85793/1/license.txteb34b1cf90b7e1103fc9dfd26be24b4aMD51ORIGINAL1128400916.2024.pdf1128400916.2024.pdfTesis de Maestría en Ingeniería - Analíticaapplication/pdf524367https://repositorio.unal.edu.co/bitstream/unal/85793/2/1128400916.2024.pdfb8f3f112e7365a4a3d6a657f8fa32befMD52THUMBNAIL1128400916.2024.pdf.jpg1128400916.2024.pdf.jpgGenerated Thumbnailimage/jpeg5352https://repositorio.unal.edu.co/bitstream/unal/85793/3/1128400916.2024.pdf.jpg21db169f4c0ca26c0e9beba3fbb09482MD53unal/85793oai:repositorio.unal.edu.co:unal/857932024-03-11 23:04:14.018Repositorio Institucional Universidad Nacional de Colombiarepositorio_nal@unal.edu.coUEFSVEUgMS4gVMOJUk1JTk9TIERFIExBIExJQ0VOQ0lBIFBBUkEgUFVCTElDQUNJw5NOIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KCkxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgbG9zIGRlcmVjaG9zIHBhdHJpbW9uaWFsZXMgZGUgYXV0b3IsIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEsIGxpbWl0YWRhIHkgZ3JhdHVpdGEgc29icmUgbGEgb2JyYSBxdWUgc2UgaW50ZWdyYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsLCBiYWpvIGxvcyBzaWd1aWVudGVzIHTDqXJtaW5vczoKCgphKQlMb3MgYXV0b3JlcyB5L28gbG9zIHRpdHVsYXJlcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBzb2JyZSBsYSBvYnJhIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEgcGFyYSByZWFsaXphciBsb3Mgc2lndWllbnRlcyBhY3RvcyBzb2JyZSBsYSBvYnJhOiBpKSByZXByb2R1Y2lyIGxhIG9icmEgZGUgbWFuZXJhIGRpZ2l0YWwsIHBlcm1hbmVudGUgbyB0ZW1wb3JhbCwgaW5jbHV5ZW5kbyBlbCBhbG1hY2VuYW1pZW50byBlbGVjdHLDs25pY28sIGFzw60gY29tbyBjb252ZXJ0aXIgZWwgZG9jdW1lbnRvIGVuIGVsIGN1YWwgc2UgZW5jdWVudHJhIGNvbnRlbmlkYSBsYSBvYnJhIGEgY3VhbHF1aWVyIG1lZGlvIG8gZm9ybWF0byBleGlzdGVudGUgYSBsYSBmZWNoYSBkZSBsYSBzdXNjcmlwY2nDs24gZGUgbGEgcHJlc2VudGUgbGljZW5jaWEsIHkgaWkpIGNvbXVuaWNhciBhbCBww7pibGljbyBsYSBvYnJhIHBvciBjdWFscXVpZXIgbWVkaW8gbyBwcm9jZWRpbWllbnRvLCBlbiBtZWRpb3MgYWzDoW1icmljb3MgbyBpbmFsw6FtYnJpY29zLCBpbmNsdXllbmRvIGxhIHB1ZXN0YSBhIGRpc3Bvc2ljacOzbiBlbiBhY2Nlc28gYWJpZXJ0by4gQWRpY2lvbmFsIGEgbG8gYW50ZXJpb3IsIGVsIGF1dG9yIHkvbyB0aXR1bGFyIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgcGFyYSBxdWUsIGVuIGxhIHJlcHJvZHVjY2nDs24geSBjb211bmljYWNpw7NuIGFsIHDDumJsaWNvIHF1ZSBsYSBVbml2ZXJzaWRhZCByZWFsaWNlIHNvYnJlIGxhIG9icmEsIGhhZ2EgbWVuY2nDs24gZGUgbWFuZXJhIGV4cHJlc2EgYWwgdGlwbyBkZSBsaWNlbmNpYSBDcmVhdGl2ZSBDb21tb25zIGJham8gbGEgY3VhbCBlbCBhdXRvciB5L28gdGl0dWxhciBkZXNlYSBvZnJlY2VyIHN1IG9icmEgYSBsb3MgdGVyY2Vyb3MgcXVlIGFjY2VkYW4gYSBkaWNoYSBvYnJhIGEgdHJhdsOpcyBkZWwgUmVwb3NpdG9yaW8gSW5zdGl0dWNpb25hbCwgY3VhbmRvIHNlYSBlbCBjYXNvLiBFbCBhdXRvciB5L28gdGl0dWxhciBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBwb2Ryw6EgZGFyIHBvciB0ZXJtaW5hZGEgbGEgcHJlc2VudGUgbGljZW5jaWEgbWVkaWFudGUgc29saWNpdHVkIGVsZXZhZGEgYSBsYSBEaXJlY2Npw7NuIE5hY2lvbmFsIGRlIEJpYmxpb3RlY2FzIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLiAKCmIpIAlMb3MgYXV0b3JlcyB5L28gdGl0dWxhcmVzIGRlIGxvcyBkZXJlY2hvcyBwYXRyaW1vbmlhbGVzIGRlIGF1dG9yIHNvYnJlIGxhIG9icmEgY29uZmllcmVuIGxhIGxpY2VuY2lhIHNlw7FhbGFkYSBlbiBlbCBsaXRlcmFsIGEpIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8gcG9yIGVsIHRpZW1wbyBkZSBwcm90ZWNjacOzbiBkZSBsYSBvYnJhIGVuIHRvZG9zIGxvcyBwYcOtc2VzIGRlbCBtdW5kbywgZXN0byBlcywgc2luIGxpbWl0YWNpw7NuIHRlcnJpdG9yaWFsIGFsZ3VuYS4KCmMpCUxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBtYW5pZmllc3RhbiBlc3RhciBkZSBhY3VlcmRvIGNvbiBxdWUgbGEgcHJlc2VudGUgbGljZW5jaWEgc2Ugb3RvcmdhIGEgdMOtdHVsbyBncmF0dWl0bywgcG9yIGxvIHRhbnRvLCByZW51bmNpYW4gYSByZWNpYmlyIGN1YWxxdWllciByZXRyaWJ1Y2nDs24gZWNvbsOzbWljYSBvIGVtb2x1bWVudG8gYWxndW5vIHBvciBsYSBwdWJsaWNhY2nDs24sIGRpc3RyaWJ1Y2nDs24sIGNvbXVuaWNhY2nDs24gcMO6YmxpY2EgeSBjdWFscXVpZXIgb3RybyB1c28gcXVlIHNlIGhhZ2EgZW4gbG9zIHTDqXJtaW5vcyBkZSBsYSBwcmVzZW50ZSBsaWNlbmNpYSB5IGRlIGxhIGxpY2VuY2lhIENyZWF0aXZlIENvbW1vbnMgY29uIHF1ZSBzZSBwdWJsaWNhLgoKZCkJUXVpZW5lcyBmaXJtYW4gZWwgcHJlc2VudGUgZG9jdW1lbnRvIGRlY2xhcmFuIHF1ZSBwYXJhIGxhIGNyZWFjacOzbiBkZSBsYSBvYnJhLCBubyBzZSBoYW4gdnVsbmVyYWRvIGxvcyBkZXJlY2hvcyBkZSBwcm9waWVkYWQgaW50ZWxlY3R1YWwsIGluZHVzdHJpYWwsIG1vcmFsZXMgeSBwYXRyaW1vbmlhbGVzIGRlIHRlcmNlcm9zLiBEZSBvdHJhIHBhcnRlLCAgcmVjb25vY2VuIHF1ZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhY3TDumEgY29tbyB1biB0ZXJjZXJvIGRlIGJ1ZW5hIGZlIHkgc2UgZW5jdWVudHJhIGV4ZW50YSBkZSBjdWxwYSBlbiBjYXNvIGRlIHByZXNlbnRhcnNlIGFsZ8O6biB0aXBvIGRlIHJlY2xhbWFjacOzbiBlbiBtYXRlcmlhIGRlIGRlcmVjaG9zIGRlIGF1dG9yIG8gcHJvcGllZGFkIGludGVsZWN0dWFsIGVuIGdlbmVyYWwuIFBvciBsbyB0YW50bywgbG9zIGZpcm1hbnRlcyAgYWNlcHRhbiBxdWUgY29tbyB0aXR1bGFyZXMgw7puaWNvcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciwgYXN1bWlyw6FuIHRvZGEgbGEgcmVzcG9uc2FiaWxpZGFkIGNpdmlsLCBhZG1pbmlzdHJhdGl2YSB5L28gcGVuYWwgcXVlIHB1ZWRhIGRlcml2YXJzZSBkZSBsYSBwdWJsaWNhY2nDs24gZGUgbGEgb2JyYS4gIAoKZikJQXV0b3JpemFuIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgaW5jbHVpciBsYSBvYnJhIGVuIGxvcyBhZ3JlZ2Fkb3JlcyBkZSBjb250ZW5pZG9zLCBidXNjYWRvcmVzIGFjYWTDqW1pY29zLCBtZXRhYnVzY2Fkb3Jlcywgw61uZGljZXMgeSBkZW3DoXMgbWVkaW9zIHF1ZSBzZSBlc3RpbWVuIG5lY2VzYXJpb3MgcGFyYSBwcm9tb3ZlciBlbCBhY2Nlc28geSBjb25zdWx0YSBkZSBsYSBtaXNtYS4gCgpnKQlFbiBlbCBjYXNvIGRlIGxhcyB0ZXNpcyBjcmVhZGFzIHBhcmEgb3B0YXIgZG9ibGUgdGl0dWxhY2nDs24sIGxvcyBmaXJtYW50ZXMgc2Vyw6FuIGxvcyByZXNwb25zYWJsZXMgZGUgY29tdW5pY2FyIGEgbGFzIGluc3RpdHVjaW9uZXMgbmFjaW9uYWxlcyBvIGV4dHJhbmplcmFzIGVuIGNvbnZlbmlvLCBsYXMgbGljZW5jaWFzIGRlIGFjY2VzbyBhYmllcnRvIENyZWF0aXZlIENvbW1vbnMgeSBhdXRvcml6YWNpb25lcyBhc2lnbmFkYXMgYSBzdSBvYnJhIHBhcmEgbGEgcHVibGljYWNpw7NuIGVuIGVsIFJlcG9zaXRvcmlvIEluc3RpdHVjaW9uYWwgVU5BTCBkZSBhY3VlcmRvIGNvbiBsYXMgZGlyZWN0cmljZXMgZGUgbGEgUG9sw610aWNhIEdlbmVyYWwgZGUgbGEgQmlibGlvdGVjYSBEaWdpdGFsLgoKCmgpCVNlIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgY29tbyByZXNwb25zYWJsZSBkZWwgdHJhdGFtaWVudG8gZGUgZGF0b3MgcGVyc29uYWxlcywgZGUgYWN1ZXJkbyBjb24gbGEgbGV5IDE1ODEgZGUgMjAxMiBlbnRlbmRpZW5kbyBxdWUgc2UgZW5jdWVudHJhbiBiYWpvIG1lZGlkYXMgcXVlIGdhcmFudGl6YW4gbGEgc2VndXJpZGFkLCBjb25maWRlbmNpYWxpZGFkIGUgaW50ZWdyaWRhZCwgeSBzdSB0cmF0YW1pZW50byB0aWVuZSB1bmEgZmluYWxpZGFkIGhpc3TDs3JpY2EsIGVzdGFkw61zdGljYSBvIGNpZW50w61maWNhIHNlZ8O6biBsbyBkaXNwdWVzdG8gZW4gbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMuCgoKClBBUlRFIDIuIEFVVE9SSVpBQ0nDk04gUEFSQSBQVUJMSUNBUiBZIFBFUk1JVElSIExBIENPTlNVTFRBIFkgVVNPIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KClNlIGF1dG9yaXphIGxhIHB1YmxpY2FjacOzbiBlbGVjdHLDs25pY2EsIGNvbnN1bHRhIHkgdXNvIGRlIGxhIG9icmEgcG9yIHBhcnRlIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgZGUgc3VzIHVzdWFyaW9zIGRlIGxhIHNpZ3VpZW50ZSBtYW5lcmE6CgphLglDb25jZWRvIGxpY2VuY2lhIGVuIGxvcyB0w6lybWlub3Mgc2XDsWFsYWRvcyBlbiBsYSBwYXJ0ZSAxIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8sIGNvbiBlbCBvYmpldGl2byBkZSBxdWUgbGEgb2JyYSBlbnRyZWdhZGEgc2VhIHB1YmxpY2FkYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgcHVlc3RhIGEgZGlzcG9zaWNpw7NuIGVuIGFjY2VzbyBhYmllcnRvIHBhcmEgc3UgY29uc3VsdGEgcG9yIGxvcyB1c3VhcmlvcyBkZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSAgYSB0cmF2w6lzIGRlIGludGVybmV0LgoKCgpQQVJURSAzIEFVVE9SSVpBQ0nDk04gREUgVFJBVEFNSUVOVE8gREUgREFUT1MgUEVSU09OQUxFUy4KCkxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLCBjb21vIHJlc3BvbnNhYmxlIGRlbCBUcmF0YW1pZW50byBkZSBEYXRvcyBQZXJzb25hbGVzLCBpbmZvcm1hIHF1ZSBsb3MgZGF0b3MgZGUgY2Fyw6FjdGVyIHBlcnNvbmFsIHJlY29sZWN0YWRvcyBtZWRpYW50ZSBlc3RlIGZvcm11bGFyaW8sIHNlIGVuY3VlbnRyYW4gYmFqbyBtZWRpZGFzIHF1ZSBnYXJhbnRpemFuIGxhIHNlZ3VyaWRhZCwgY29uZmlkZW5jaWFsaWRhZCBlIGludGVncmlkYWQgeSBzdSB0cmF0YW1pZW50byBzZSByZWFsaXphIGRlIGFjdWVyZG8gYWwgY3VtcGxpbWllbnRvIG5vcm1hdGl2byBkZSBsYSBMZXkgMTU4MSBkZSAyMDEyIHkgZGUgbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMgZGUgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEuIFB1ZWRlIGVqZXJjZXIgc3VzIGRlcmVjaG9zIGNvbW8gdGl0dWxhciBhIGNvbm9jZXIsIGFjdHVhbGl6YXIsIHJlY3RpZmljYXIgeSByZXZvY2FyIGxhcyBhdXRvcml6YWNpb25lcyBkYWRhcyBhIGxhcyBmaW5hbGlkYWRlcyBhcGxpY2FibGVzIGEgdHJhdsOpcyBkZSBsb3MgY2FuYWxlcyBkaXNwdWVzdG9zIHkgZGlzcG9uaWJsZXMgZW4gd3d3LnVuYWwuZWR1LmNvIG8gZS1tYWlsOiBwcm90ZWNkYXRvc19uYUB1bmFsLmVkdS5jbyIKClRlbmllbmRvIGVuIGN1ZW50YSBsbyBhbnRlcmlvciwgYXV0b3Jpem8gZGUgbWFuZXJhIHZvbHVudGFyaWEsIHByZXZpYSwgZXhwbMOtY2l0YSwgaW5mb3JtYWRhIGUgaW5lcXXDrXZvY2EgYSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhIHRyYXRhciBsb3MgZGF0b3MgcGVyc29uYWxlcyBkZSBhY3VlcmRvIGNvbiBsYXMgZmluYWxpZGFkZXMgZXNwZWPDrWZpY2FzIHBhcmEgZWwgZGVzYXJyb2xsbyB5IGVqZXJjaWNpbyBkZSBsYXMgZnVuY2lvbmVzIG1pc2lvbmFsZXMgZGUgZG9jZW5jaWEsIGludmVzdGlnYWNpw7NuIHkgZXh0ZW5zacOzbiwgYXPDrSBjb21vIGxhcyByZWxhY2lvbmVzIGFjYWTDqW1pY2FzLCBsYWJvcmFsZXMsIGNvbnRyYWN0dWFsZXMgeSB0b2RhcyBsYXMgZGVtw6FzIHJlbGFjaW9uYWRhcyBjb24gZWwgb2JqZXRvIHNvY2lhbCBkZSBsYSBVbml2ZXJzaWRhZC4gCgo=