ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes
Accurate mortality prediction allows Intensive Care Units (ICUs) to adequately benchmark clinical practice and identify patients with unexpected outcomes. Traditionally, simple statistical models have been used to assess patient death risk, many times with sub-optimal performance. On the other hand...
- Autores:
-
Caicedo-Torres, William
Gutierrez, Jairo
- Tipo de recurso:
- Fecha de publicación:
- 2020
- Institución:
- Universidad Tecnológica de Bolívar
- Repositorio:
- Repositorio Institucional UTB
- Idioma:
- eng
- OAI Identifier:
- oai:repositorio.utb.edu.co:20.500.12585/11120
- Acceso en línea:
- https://hdl.handle.net/20.500.12585/11120
https://doi.org/10.1016/j.eswa.2022.117190
- Palabra clave:
- Deep learning
MIMIC-III
Clinical notes
Shapley Value
LEMB
- Rights
- openAccess
- License
- http://creativecommons.org/licenses/by-nc-nd/4.0/
| id |
UTB2_0be856db31eae8e5a2cf3f0cbc9fa9ac |
|---|---|
| oai_identifier_str |
oai:repositorio.utb.edu.co:20.500.12585/11120 |
| network_acronym_str |
UTB2 |
| network_name_str |
Repositorio Institucional UTB |
| repository_id_str |
|
| dc.title.spa.fl_str_mv |
ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes |
| title |
ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes |
| spellingShingle |
ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes Deep learning MIMIC-III Clinical notes Shapley Value LEMB |
| title_short |
ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes |
| title_full |
ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes |
| title_fullStr |
ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes |
| title_full_unstemmed |
ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes |
| title_sort |
ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes |
| dc.creator.fl_str_mv |
Caicedo-Torres, William Gutierrez, Jairo |
| dc.contributor.author.none.fl_str_mv |
Caicedo-Torres, William Gutierrez, Jairo |
| dc.subject.keywords.spa.fl_str_mv |
Deep learning MIMIC-III Clinical notes Shapley Value |
| topic |
Deep learning MIMIC-III Clinical notes Shapley Value LEMB |
| dc.subject.armarc.none.fl_str_mv |
LEMB |
| description |
Accurate mortality prediction allows Intensive Care Units (ICUs) to adequately benchmark clinical practice and identify patients with unexpected outcomes. Traditionally, simple statistical models have been used to assess patient death risk, many times with sub-optimal performance. On the other hand deep learning holds promise to positively impact clinical practice by leveraging medical data to assist diagnosis and prediction, including mortality prediction. However, as the question of whether powerful Deep Learning models attend correlations backed by sound medical knowledge when generating predictions remains open, additional interpretability tools are needed to foster trust and encourage the use of AI by clinicians. In this work we show a Deep Learning model trained on MIMIC-III to predict mortality using raw nursing notes, together with visual explanations for word importance. Our model reaches a ROC of 0.8629 (±0.0058), outperforming the traditional SAPS-II score and providing enhanced interpretability when compared with similar Deep Learning approaches. |
| publishDate |
2020 |
| dc.date.issued.none.fl_str_mv |
2020-05-20 |
| dc.date.accessioned.none.fl_str_mv |
2022-09-29T13:22:47Z |
| dc.date.available.none.fl_str_mv |
2022-09-29T13:22:47Z |
| dc.date.submitted.none.fl_str_mv |
2022-09-28 |
| dc.type.driver.spa.fl_str_mv |
info:eu-repo/semantics/article |
| dc.type.hasversion.spa.fl_str_mv |
info:eu-repo/semantics/restrictedAccess |
| dc.type.spa.spa.fl_str_mv |
http://purl.org/coar/resource_type/c_2df8fbb1 |
| dc.identifier.citation.spa.fl_str_mv |
Caicedo, William & Gutierrez, Jairo. (2020). ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes. |
| dc.identifier.uri.none.fl_str_mv |
https://hdl.handle.net/20.500.12585/11120 |
| dc.identifier.doi.none.fl_str_mv |
https://doi.org/10.1016/j.eswa.2022.117190 |
| dc.identifier.instname.spa.fl_str_mv |
Universidad Tecnológica de Bolívar |
| dc.identifier.reponame.spa.fl_str_mv |
Repositorio Universidad Tecnológica de Bolívar |
| identifier_str_mv |
Caicedo, William & Gutierrez, Jairo. (2020). ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes. Universidad Tecnológica de Bolívar Repositorio Universidad Tecnológica de Bolívar |
| url |
https://hdl.handle.net/20.500.12585/11120 https://doi.org/10.1016/j.eswa.2022.117190 |
| dc.language.iso.spa.fl_str_mv |
eng |
| language |
eng |
| dc.rights.coar.fl_str_mv |
http://purl.org/coar/access_right/c_abf2 |
| dc.rights.uri.*.fl_str_mv |
http://creativecommons.org/licenses/by-nc-nd/4.0/ |
| dc.rights.accessrights.spa.fl_str_mv |
info:eu-repo/semantics/openAccess |
| dc.rights.cc.*.fl_str_mv |
Attribution-NonCommercial-NoDerivatives 4.0 Internacional |
| rights_invalid_str_mv |
http://creativecommons.org/licenses/by-nc-nd/4.0/ Attribution-NonCommercial-NoDerivatives 4.0 Internacional http://purl.org/coar/access_right/c_abf2 |
| eu_rights_str_mv |
openAccess |
| dc.format.extent.none.fl_str_mv |
32 Páginas |
| dc.format.mimetype.spa.fl_str_mv |
application/pdf |
| dc.publisher.place.spa.fl_str_mv |
Cartagena de Indias |
| dc.source.spa.fl_str_mv |
ScienceDirect - Elsevier - Expert Systems with Applications Vol. 202 (2022) |
| institution |
Universidad Tecnológica de Bolívar |
| bitstream.url.fl_str_mv |
https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/1/ISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/2/license_rdf https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/3/license.txt https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/4/ISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf.txt https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/5/ISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf.jpg |
| bitstream.checksum.fl_str_mv |
848bd2300a24b95eed83b914c9bb8e24 4460e5956bc1d1639be9ae6146a50347 e20ad307a1c5f3f25af9304a7a7c86b6 45dc17210c4cb0901e99113703983e59 e1f5a1c51100daaba58d701c556455cc |
| bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 MD5 MD5 MD5 |
| repository.name.fl_str_mv |
Repositorio Institucional UTB |
| repository.mail.fl_str_mv |
repositorioutb@utb.edu.co |
| _version_ |
1814021707456839680 |
| spelling |
Caicedo-Torres, William865cbcee-ba06-417f-a6ae-50ba943243e3Gutierrez, Jairo32d064db-e471-4a23-9512-7b634356d9c92022-09-29T13:22:47Z2022-09-29T13:22:47Z2020-05-202022-09-28Caicedo, William & Gutierrez, Jairo. (2020). ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notes.https://hdl.handle.net/20.500.12585/11120https://doi.org/10.1016/j.eswa.2022.117190Universidad Tecnológica de BolívarRepositorio Universidad Tecnológica de BolívarAccurate mortality prediction allows Intensive Care Units (ICUs) to adequately benchmark clinical practice and identify patients with unexpected outcomes. Traditionally, simple statistical models have been used to assess patient death risk, many times with sub-optimal performance. On the other hand deep learning holds promise to positively impact clinical practice by leveraging medical data to assist diagnosis and prediction, including mortality prediction. However, as the question of whether powerful Deep Learning models attend correlations backed by sound medical knowledge when generating predictions remains open, additional interpretability tools are needed to foster trust and encourage the use of AI by clinicians. In this work we show a Deep Learning model trained on MIMIC-III to predict mortality using raw nursing notes, together with visual explanations for word importance. Our model reaches a ROC of 0.8629 (±0.0058), outperforming the traditional SAPS-II score and providing enhanced interpretability when compared with similar Deep Learning approaches.32 Páginasapplication/pdfenghttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessAttribution-NonCommercial-NoDerivatives 4.0 Internacionalhttp://purl.org/coar/access_right/c_abf2ScienceDirect - Elsevier - Expert Systems with Applications Vol. 202 (2022)ISeeU2: Visually Interpretable ICU mortality prediction using deep learning and free-text medical notesinfo:eu-repo/semantics/articleinfo:eu-repo/semantics/restrictedAccesshttp://purl.org/coar/resource_type/c_2df8fbb1Deep learningMIMIC-IIIClinical notesShapley ValueLEMBCartagena de IndiasG. Grasselli, A. Pesenti, M. Cecconi, Critical Care Utilization for the COVID-19 Outbreak in Lombardy, Italy, JAMA (2020). doi:10.1001/ jama.2020.4031.E. J. Emanuel, G. Persad, R. Upshur, B. Thome, M. Parker, A. Glick man, C. Zhang, C. Boyle, M. Smith, J. P. Phillips, Fair Allocatio Scarce Medical Resources in the Time of Covid-19, New England Journal of Medicine (2020). doi:10.1056/nejmsb2005114.A. G. Rapsang, D. C. Shyam, Scoring systems in the intensive care unit: A compendium, Indian Journal of Critical Care Medicine : Peer reviewed, Official Publication of Indian Society of Critical Care Medicine 18 (4) (2014) 220–228. doi:10.4103/0972-5229.130573. URL http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4033855/S. Purushotham, C. Meng, Z. Che, Y. Liu, Benchmarking Deep Learning Models on Large Healthcare Datasets, Journal of Biomedical Informat ics (2018). doi:https://doi.org/10.1016/j.jbi.2018.04.007. URL http://www.sciencedirect.com/science/article/pii/ S1532046418300716W. Caicedo-Torres, J. Gutierrez, ISeeU: Visually interpretable deep learning for mortality prediction inside the ICU, Journal of Biomedical Informatics 98 (2019) 103269. doi:10.1016/J.JBI.2019.103269. URL https://www.sciencedirect.com/science/article/pii/ S1532046419301881?dgcid=authorD. Shen, G. Wu, H.-I. Suk, Deep Learning in Medical Image Analysis, Annual Review of Biomedical Engineering 19 (1) (2017) null. doi: 10.1146/annurev-bioeng-071516-044442. URL http://dx.doi.org/10.1146/annurev-bioeng-071516-044442B. Shickel, P. J. Tighe, A. Bihorac, P. Rashidi, Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis., IEEE journal of biomedical and health infor matics 22 (5) (2018) 1589–1604. doi:10.1109/JBHI.2017.27P. Grnarova, F. Schmidt, S. L. Hyland, C. Eickhoff, Neural Document Embeddings for Intensive Care Patient Mortality Prediction, CoRR abs/1612.0 (2016).G. F. Cooper, C. F. Aliferis, R. Ambrosino, J. Aronis, B. G. Buchanan, R. Caruana, M. J. Fine, C. Glymour, G. Gordon, B. H. Hanusa, J. E. Janosky, C. Meek, T. Mitchell, T. Richardson, P. Spirtes, An evaluation of machine-learning methods for predicting pneumonia mortality, Ar tificial Intelligence in Medicine (1997). doi:10.1016/S0933-3657(96) 00367-3.A. E. W. Johnson, T. J. Pollard, L. Shen, L.-W. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. A. Celi, R. G. Mark, MIMIC III, a freely accessible critical care database., Sci Data 3 (2016) 160035. doi:10.1038/sdata.2016.35.Y. Jo, L. Lee, S. Palaskar, Combining LSTM and Latent Topic Modeling for Mortality Prediction, ArXiv abs/1709.0 (2017).M. Sushil, S. Suster, K. Luyckx, W. Daelemans, Patient representation ˇ learning and interpretable evaluation using clinical notes, Journal of Biomedical Informatics (2018). arXiv:1807.01395, doi:10.1016/j. jbi.2018.06.016.Y. Si, K. Roberts, Deep Patient Representation of Clinical Notes via Multi-Task Learning for Mortality Prediction., AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Trans lational Science (2019).M. Jin, M. T. Bahadori, A. Colak, P. Bhatia, B. Celikkaya, R. Bhakta, S. Senthivel, M. Khalilia, D. Navarro, B. Zhang, T. Doman, A. Ravi, M. Liger, T. Kass-hout, Improving Hospital Mortality Prediction with Medical Named Entities and Multimodal Learning (2018). arXiv:1811. 12276.Y. LeCun, L. Bottou, Y. Bengio, Haffner, Gradient-Based Learning Ap plied to Document Recognition, in: Proceedings of the IEEE, Vol. 86, 1998, pp. 2278–2324.I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016L. S. Shapley, A Value for n-Person Games, in: H. W. Kuhn, A. W. Tucker (Eds.), Contributions to the Theory of Games II, Princeton Uni versity Press, Princeton, 1953, pp. 307–317E. Strumbelj, I. Kononenko, S. Wrobel, An Efficient Explanation of Indi vidual Classifications using Game Theory, Journal of Machine Learning Research (2010). arXiv:1606.05386, doi:10.1145/2858036.2858529.A. Shrikumar, P. Greenside, A. Kundaje, Learning Important Features Through Propagating Activation Differences, CoRR abs/1704.0 (2017). arXiv:1704.02685. URL http://arxiv.org/abs/1704.02685S. M. Lundberg, S. I. Lee, A unified approach to interpreting model predictions, in: Advances in Neural Information Processing Systems, 2017. arXiv:1705.07874.K. Simonyan, A. Vedaldi, A. Zisserman, Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps, CoRR abs/1312.6 (2013). URL http://arxiv.org/abs/1312.6034J. T. Springenberg, A. Dosovitskiy, T. Brox, M. A. Riedmiller, Striving for Simplicity: The All Convolutional Net, CoRR abs/1412.6 (2014). arXiv:1412.6806. URL http://arxiv.org/abs/1412.6806M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, L. Kaiser, M. Kudlur, J. Leven berg, D. Man, R. Monga, S. Moore, D. Murray, J. Shlens, B. Steiner, I. Sutskever, P. Tucker, V. Vanhoucke, V. Vasudevan, O. Vinyals, P. Warden, M. Wicke, Y. Yu, X. Zheng, TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems, None 1 (212) (2015) 19. arXiv:1603.04467, doi:10.1038/nn.3331. URL http://download.tensorflow.org/paper/whitepaper2015. pdfD. Kingma, J. Ba, Adam: A Method for Stochastic Optimization, ArXiv e-prints (dec 2014). arXiv:1412.6980E. Loper, S. Bird, NLTK, 2002. doi:10.3115/1118108.1118117J. R. Gall, S. Lemeshow, F. Saulnier, A New Simplified Acute Physiol ogy Score (SAPS II) Based on a European/North American Multicenter Study, JAMA: The Journal of the American Medical Association (1993). arXiv:0402594v3, doi:10.1001/jama.1993.03510240069035.A. E. Johnson, D. J. Stone, L. A. Celi, T. J. Pollard, The MIMIC Code Repository: Enabling reproducibility in critical care research, Journal of the American Medical Informatics Association (2018). doi:10.1093/ jamia/ocx084.S. Hochreiter, J. Schmidhuber, Long Short-Term Memory, Neural Com put. 9 (8) (1997) 1735–1780. doi:10.1162/neco.1997.9.8.1735. URL http://dx.doi.org/10.1162/neco.1997.9.8.1735Z. C. Lipton, The Mythos of Model Interpretability, ICML Workshop on Human Interpretability in Machine Learning abs/1606.0 (2016) 96–100. arXiv:arXiv:1606.03490v1. URL http://arxiv.org/abs/1606.03490Z. Che, S. Purushotham, K. Cho, D. Sontag, Y. Liu, Recurrent Neu ral Networks for Multivariate Time Series with Missing Values, CoRR abs/1606.0 (2016). URL http://arxiv.org/abs/1606.01865Z. C. Lipton, D. Kale, R. Wetzel, Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series, in: F. Doshi-Velez, J. Fackler, D. Kale, B. Wallace, J. Weins (Eds.), Proceedings of the 1st Machine Learning for Healthcare Conference, Vol. 56 of Proceedings of Machine Learning Research, PMLR, North eastern University, Boston, MA, USA, 2016, pp. 253–270. URL http://proceedings.mlr.press/v56/Lipton16.htmlVol. 56 of Proceedings of Machine Learning Research, PMLR, North eastern University, Boston, MA, USA, 2016, pp. 253–270. URL http://proceedings.mlr.press/v56/Lipton16.htmlhttp://purl.org/coar/resource_type/c_2df8fbb1ORIGINALISeeU2_Visually_Interpretable_ICU_mortality_predic.pdfISeeU2_Visually_Interpretable_ICU_mortality_predic.pdfapplication/pdf8820916https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/1/ISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf848bd2300a24b95eed83b914c9bb8e24MD51CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8805https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/2/license_rdf4460e5956bc1d1639be9ae6146a50347MD52LICENSElicense.txtlicense.txttext/plain; charset=utf-83182https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/3/license.txte20ad307a1c5f3f25af9304a7a7c86b6MD53TEXTISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf.txtISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf.txtExtracted texttext/plain39232https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/4/ISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf.txt45dc17210c4cb0901e99113703983e59MD54THUMBNAILISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf.jpgISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf.jpgGenerated Thumbnailimage/jpeg5827https://repositorio.utb.edu.co/bitstream/20.500.12585/11120/5/ISeeU2_Visually_Interpretable_ICU_mortality_predic.pdf.jpge1f5a1c51100daaba58d701c556455ccMD5520.500.12585/11120oai:repositorio.utb.edu.co:20.500.12585/111202022-09-30 00:18:49.872Repositorio Institucional UTBrepositorioutb@utb.edu.coQXV0b3Jpem8gKGF1dG9yaXphbW9zKSBhIGxhIEJpYmxpb3RlY2EgZGUgbGEgSW5zdGl0dWNpw7NuIHBhcmEgcXVlIGluY2x1eWEgdW5hIGNvcGlhLCBpbmRleGUgeSBkaXZ1bGd1ZSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsLCBsYSBvYnJhIG1lbmNpb25hZGEgY29uIGVsIGZpbiBkZSBmYWNpbGl0YXIgbG9zIHByb2Nlc29zIGRlIHZpc2liaWxpZGFkIGUgaW1wYWN0byBkZSBsYSBtaXNtYSwgY29uZm9ybWUgYSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBxdWUgbWUobm9zKSBjb3JyZXNwb25kZShuKSB5IHF1ZSBpbmNsdXllbjogbGEgcmVwcm9kdWNjacOzbiwgY29tdW5pY2FjacOzbiBww7pibGljYSwgZGlzdHJpYnVjacOzbiBhbCBww7pibGljbywgdHJhbnNmb3JtYWNpw7NuLCBkZSBjb25mb3JtaWRhZCBjb24gbGEgbm9ybWF0aXZpZGFkIHZpZ2VudGUgc29icmUgZGVyZWNob3MgZGUgYXV0b3IgeSBkZXJlY2hvcyBjb25leG9zIHJlZmVyaWRvcyBlbiBhcnQuIDIsIDEyLCAzMCAobW9kaWZpY2FkbyBwb3IgZWwgYXJ0IDUgZGUgbGEgbGV5IDE1MjAvMjAxMiksIHkgNzIgZGUgbGEgbGV5IDIzIGRlIGRlIDE5ODIsIExleSA0NCBkZSAxOTkzLCBhcnQuIDQgeSAxMSBEZWNpc2nDs24gQW5kaW5hIDM1MSBkZSAxOTkzIGFydC4gMTEsIERlY3JldG8gNDYwIGRlIDE5OTUsIENpcmN1bGFyIE5vIDA2LzIwMDIgZGUgbGEgRGlyZWNjacOzbiBOYWNpb25hbCBkZSBEZXJlY2hvcyBkZSBhdXRvciwgYXJ0LiAxNSBMZXkgMTUyMCBkZSAyMDEyLCBsYSBMZXkgMTkxNSBkZSAyMDE4IHkgZGVtw6FzIG5vcm1hcyBzb2JyZSBsYSBtYXRlcmlhLgoKQWwgcmVzcGVjdG8gY29tbyBBdXRvcihlcykgbWFuaWZlc3RhbW9zIGNvbm9jZXIgcXVlOgoKLSBMYSBhdXRvcml6YWNpw7NuIGVzIGRlIGNhcsOhY3RlciBubyBleGNsdXNpdmEgeSBsaW1pdGFkYSwgZXN0byBpbXBsaWNhIHF1ZSBsYSBsaWNlbmNpYSB0aWVuZSB1bmEgdmlnZW5jaWEsIHF1ZSBubyBlcyBwZXJwZXR1YSB5IHF1ZSBlbCBhdXRvciBwdWVkZSBwdWJsaWNhciBvIGRpZnVuZGlyIHN1IG9icmEgZW4gY3VhbHF1aWVyIG90cm8gbWVkaW8sIGFzw60gY29tbyBsbGV2YXIgYSBjYWJvIGN1YWxxdWllciB0aXBvIGRlIGFjY2nDs24gc29icmUgZWwgZG9jdW1lbnRvLgoKLSBMYSBhdXRvcml6YWNpw7NuIHRlbmRyw6EgdW5hIHZpZ2VuY2lhIGRlIGNpbmNvIGHDsW9zIGEgcGFydGlyIGRlbCBtb21lbnRvIGRlIGxhIGluY2x1c2nDs24gZGUgbGEgb2JyYSBlbiBlbCByZXBvc2l0b3JpbywgcHJvcnJvZ2FibGUgaW5kZWZpbmlkYW1lbnRlIHBvciBlbCB0aWVtcG8gZGUgZHVyYWNpw7NuIGRlIGxvcyBkZXJlY2hvcyBwYXRyaW1vbmlhbGVzIGRlbCBhdXRvciB5IHBvZHLDoSBkYXJzZSBwb3IgdGVybWluYWRhIHVuYSB2ZXogZWwgYXV0b3IgbG8gbWFuaWZpZXN0ZSBwb3IgZXNjcml0byBhIGxhIGluc3RpdHVjacOzbiwgY29uIGxhIHNhbHZlZGFkIGRlIHF1ZSBsYSBvYnJhIGVzIGRpZnVuZGlkYSBnbG9iYWxtZW50ZSB5IGNvc2VjaGFkYSBwb3IgZGlmZXJlbnRlcyBidXNjYWRvcmVzIHkvbyByZXBvc2l0b3Jpb3MgZW4gSW50ZXJuZXQgbG8gcXVlIG5vIGdhcmFudGl6YSBxdWUgbGEgb2JyYSBwdWVkYSBzZXIgcmV0aXJhZGEgZGUgbWFuZXJhIGlubWVkaWF0YSBkZSBvdHJvcyBzaXN0ZW1hcyBkZSBpbmZvcm1hY2nDs24gZW4gbG9zIHF1ZSBzZSBoYXlhIGluZGV4YWRvLCBkaWZlcmVudGVzIGFsIHJlcG9zaXRvcmlvIGluc3RpdHVjaW9uYWwgZGUgbGEgSW5zdGl0dWNpw7NuLCBkZSBtYW5lcmEgcXVlIGVsIGF1dG9yKHJlcykgdGVuZHLDoW4gcXVlIHNvbGljaXRhciBsYSByZXRpcmFkYSBkZSBzdSBvYnJhIGRpcmVjdGFtZW50ZSBhIG90cm9zIHNpc3RlbWFzIGRlIGluZm9ybWFjacOzbiBkaXN0aW50b3MgYWwgZGUgbGEgSW5zdGl0dWNpw7NuIHNpIGRlc2VhIHF1ZSBzdSBvYnJhIHNlYSByZXRpcmFkYSBkZSBpbm1lZGlhdG8uCgotIExhIGF1dG9yaXphY2nDs24gZGUgcHVibGljYWNpw7NuIGNvbXByZW5kZSBlbCBmb3JtYXRvIG9yaWdpbmFsIGRlIGxhIG9icmEgeSB0b2RvcyBsb3MgZGVtw6FzIHF1ZSBzZSByZXF1aWVyYSBwYXJhIHN1IHB1YmxpY2FjacOzbiBlbiBlbCByZXBvc2l0b3Jpby4gSWd1YWxtZW50ZSwgbGEgYXV0b3JpemFjacOzbiBwZXJtaXRlIGEgbGEgaW5zdGl0dWNpw7NuIGVsIGNhbWJpbyBkZSBzb3BvcnRlIGRlIGxhIG9icmEgY29uIGZpbmVzIGRlIHByZXNlcnZhY2nDs24gKGltcHJlc28sIGVsZWN0csOzbmljbywgZGlnaXRhbCwgSW50ZXJuZXQsIGludHJhbmV0LCBvIGN1YWxxdWllciBvdHJvIGZvcm1hdG8gY29ub2NpZG8gbyBwb3IgY29ub2NlcikuCgotIExhIGF1dG9yaXphY2nDs24gZXMgZ3JhdHVpdGEgeSBzZSByZW51bmNpYSBhIHJlY2liaXIgY3VhbHF1aWVyIHJlbXVuZXJhY2nDs24gcG9yIGxvcyB1c29zIGRlIGxhIG9icmEsIGRlIGFjdWVyZG8gY29uIGxhIGxpY2VuY2lhIGVzdGFibGVjaWRhIGVuIGVzdGEgYXV0b3JpemFjacOzbi4KCi0gQWwgZmlybWFyIGVzdGEgYXV0b3JpemFjacOzbiwgc2UgbWFuaWZpZXN0YSBxdWUgbGEgb2JyYSBlcyBvcmlnaW5hbCB5IG5vIGV4aXN0ZSBlbiBlbGxhIG5pbmd1bmEgdmlvbGFjacOzbiBhIGxvcyBkZXJlY2hvcyBkZSBhdXRvciBkZSB0ZXJjZXJvcy4gRW4gY2FzbyBkZSBxdWUgZWwgdHJhYmFqbyBoYXlhIHNpZG8gZmluYW5jaWFkbyBwb3IgdGVyY2Vyb3MgZWwgbyBsb3MgYXV0b3JlcyBhc3VtZW4gbGEgcmVzcG9uc2FiaWxpZGFkIGRlbCBjdW1wbGltaWVudG8gZGUgbG9zIGFjdWVyZG9zIGVzdGFibGVjaWRvcyBzb2JyZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBsYSBvYnJhIGNvbiBkaWNobyB0ZXJjZXJvLgoKLSBGcmVudGUgYSBjdWFscXVpZXIgcmVjbGFtYWNpw7NuIHBvciB0ZXJjZXJvcywgZWwgbyBsb3MgYXV0b3JlcyBzZXLDoW4gcmVzcG9uc2FibGVzLCBlbiBuaW5nw7puIGNhc28gbGEgcmVzcG9uc2FiaWxpZGFkIHNlcsOhIGFzdW1pZGEgcG9yIGxhIGluc3RpdHVjacOzbi4KCi0gQ29uIGxhIGF1dG9yaXphY2nDs24sIGxhIGluc3RpdHVjacOzbiBwdWVkZSBkaWZ1bmRpciBsYSBvYnJhIGVuIMOtbmRpY2VzLCBidXNjYWRvcmVzIHkgb3Ryb3Mgc2lzdGVtYXMgZGUgaW5mb3JtYWNpw7NuIHF1ZSBmYXZvcmV6Y2FuIHN1IHZpc2liaWxpZGFkCgo= |