Exploring in and out-of-equilibrium learning regimes of restricted Boltzmann machines

ilustraciones

Autores:
Navas Gómez, Alfonso de Jesús
Tipo de recurso:
Fecha de publicación:
2022
Institución:
Universidad Nacional de Colombia
Repositorio:
Universidad Nacional de Colombia
Idioma:
eng
OAI Identifier:
oai:repositorio.unal.edu.co:unal/83483
Acceso en línea:
https://repositorio.unal.edu.co/handle/unal/83483
https://repositorio.unal.edu.co/
Palabra clave:
Complejidad computacional
Sistemas expertos
Computational complexity
Inteligencia artificial
Aprendizaje automatizado
Física estadística de sistemas desordenados
Máquinas de Boltzmann restringidas
Métodos de Monte-Carlo
Artificial intelligence
Statistical physics of disordered systems
Restricted Boltzmann machines
Monte-Carlo methods
Machine learning
Rights
openAccess
License
Atribución-NoComercial-SinDerivadas 4.0 Internacional
id UNACIONAL2_47bc0849e17234480508f774b4b1eead
oai_identifier_str oai:repositorio.unal.edu.co:unal/83483
network_acronym_str UNACIONAL2
network_name_str Universidad Nacional de Colombia
repository_id_str
dc.title.eng.fl_str_mv Exploring in and out-of-equilibrium learning regimes of restricted Boltzmann machines
dc.title.translated.none.fl_str_mv Explorando los regímenes de aprendizaje dentro y fuera del equilibrio de las máquinas de Boltzmann restringidas
title Exploring in and out-of-equilibrium learning regimes of restricted Boltzmann machines
spellingShingle Exploring in and out-of-equilibrium learning regimes of restricted Boltzmann machines
Complejidad computacional
Sistemas expertos
Computational complexity
Inteligencia artificial
Aprendizaje automatizado
Física estadística de sistemas desordenados
Máquinas de Boltzmann restringidas
Métodos de Monte-Carlo
Artificial intelligence
Statistical physics of disordered systems
Restricted Boltzmann machines
Monte-Carlo methods
Machine learning
title_short Exploring in and out-of-equilibrium learning regimes of restricted Boltzmann machines
title_full Exploring in and out-of-equilibrium learning regimes of restricted Boltzmann machines
title_fullStr Exploring in and out-of-equilibrium learning regimes of restricted Boltzmann machines
title_full_unstemmed Exploring in and out-of-equilibrium learning regimes of restricted Boltzmann machines
title_sort Exploring in and out-of-equilibrium learning regimes of restricted Boltzmann machines
dc.creator.fl_str_mv Navas Gómez, Alfonso de Jesús
dc.contributor.advisor.none.fl_str_mv Giraldo Gallo, José Jairo
Seoane Bartolomé, Beatriz
dc.contributor.author.none.fl_str_mv Navas Gómez, Alfonso de Jesús
dc.subject.lemb.spa.fl_str_mv Complejidad computacional
Sistemas expertos
topic Complejidad computacional
Sistemas expertos
Computational complexity
Inteligencia artificial
Aprendizaje automatizado
Física estadística de sistemas desordenados
Máquinas de Boltzmann restringidas
Métodos de Monte-Carlo
Artificial intelligence
Statistical physics of disordered systems
Restricted Boltzmann machines
Monte-Carlo methods
Machine learning
dc.subject.lemb.eng.fl_str_mv Computational complexity
dc.subject.proposal.spa.fl_str_mv Inteligencia artificial
Aprendizaje automatizado
Física estadística de sistemas desordenados
Máquinas de Boltzmann restringidas
Métodos de Monte-Carlo
dc.subject.proposal.eng.fl_str_mv Artificial intelligence
Statistical physics of disordered systems
Restricted Boltzmann machines
Monte-Carlo methods
Machine learning
description ilustraciones
publishDate 2022
dc.date.issued.none.fl_str_mv 2022-11-15
dc.date.accessioned.none.fl_str_mv 2023-02-15T15:50:31Z
dc.date.available.none.fl_str_mv 2023-02-15T15:50:31Z
dc.type.spa.fl_str_mv Trabajo de grado - Maestría
dc.type.driver.spa.fl_str_mv info:eu-repo/semantics/masterThesis
dc.type.version.spa.fl_str_mv info:eu-repo/semantics/acceptedVersion
dc.type.content.spa.fl_str_mv Text
dc.type.redcol.spa.fl_str_mv http://purl.org/redcol/resource_type/TM
status_str acceptedVersion
dc.identifier.uri.none.fl_str_mv https://repositorio.unal.edu.co/handle/unal/83483
dc.identifier.instname.spa.fl_str_mv Universidad Nacional de Colombia
dc.identifier.reponame.spa.fl_str_mv Repositorio Institucional Universidad Nacional de Colombia
dc.identifier.repourl.spa.fl_str_mv https://repositorio.unal.edu.co/
url https://repositorio.unal.edu.co/handle/unal/83483
https://repositorio.unal.edu.co/
identifier_str_mv Universidad Nacional de Colombia
Repositorio Institucional Universidad Nacional de Colombia
dc.language.iso.spa.fl_str_mv eng
language eng
dc.relation.references.spa.fl_str_mv D. H. Ackley, G. E. Hinton, and T. J. Sejnowski. A learning algorithm for boltzmann machines. Cognitive science, 9(1):147–169, 1985.
D. J. Amit and V. Martin-Mayor. Field theory, the renormalization group, and critical phenomena: graphs to computers. World Scientific Publishing Company, 2005.
A. Bakk and J. S. Høye. One-dimensional ising model applied to protein folding. Physica A: Statistical Mechanics and its Applications, 323:504–518, 2003.
H. Ballesteros and V. Martín-Mayor. Test for random number generators: Schwinger-Dyson equations for the ising model. Physical Review E, 58(5):6787, 1998.
S. Behnel, R. Bradshaw, C. Citro, L. Dalcin, D. S. Seljebotn, and K. Smith. Cython: The best of both worlds. Computing in Science & Engineering, 13(2):31–39, 2011.
N. Béreux, A. Decelle, C. Furtlehner, and B. Seoane. Learning a restricted Boltzmann machine using biased Monte Carlo sampling. arXiv preprint arXiv:2206.01310, 2022.
C. M. Bishop. Pattern Recognition and Machine Learning. Springer, New York, 2006.
B. Bravi, J. Tubiana, S. Cocco, R. Monasson, T. Mora, and A. M. Walczak. Rbm-mhc: A semi-supervised machine-learning method for sample-specific prediction of antigen presentation by hla-i alleles. Cell systems, 12(2):195–202, 2021.
L. Brocchieri and S. Karlin. Protein length in eukaryotic and prokaryotic proteomes. Nucleic acids research, 33(10):3390–3400, 2005.
G. Cossu, L. Del Debbio, T. Giani, A. Khamseh, and M. Wilson. Machine learning determination of dynamical parameters: The ising model case. Physical Review B, 100(6):064304, 2019.
A. Decelle. TorchRBM. https://github.com/AurelienDecelle/TorchRBM, 2021. Accessed: 20-07-2022.
A. Decelle and C. Furtlehner. Restricted Boltzmann machine: Recent advances and mean-field theory. Chinese Physics B, 30(4):040202, 2021.
A. Decelle, C. Furtlehner, and B. Seoane. Equilibrium and non-equilibrium regimes in the learning of restricted boltzmann machines. arXiv preprint arXiv:2105.13889, 2021.
A. Fischer and C. Igel. An introduction to restricted boltzmann machines. In Iberoamerican congress on pattern recognition, pages 14–36. Springer, 2012.
M. Harsh, J. Tubiana, S. Cocco, and R. Monasson. ‘place-cell’ emergence and learning of invariant data with restricted boltzmann machines: breaking and dynamical restoration of continuous symmetries in the weight space. Journal of Physics A: Mathematical and Theoretical, 53(17):174002, 2020.
W. K. Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika, 57(1):97–109, 1953.
T. L. Hill. Generalization of the one-dimensional Ising model applicable to helix transitions in nucleic acids and proteins. The Journal of Chemical Physics, 30(2):383–387, 1959.
G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):1771–1800, 2002.
G. E. Hinton. A practical guide to training restricted Boltzmann machines. In Neural networks: Tricks of the trade, pages 599–619. Springer, 2012.
G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504–507, 2006.
R. D. Hjelm, V. D. Calhoun, R. Salakhutdinov, E. A. Allen, T. Adali, and S. M. Plis. Restricted boltzmann machines for neuroimaging: an application in identifying intrinsic networks. NeuroImage, 96:245–260, 2014.
N. Le Roux and Y. Bengio. Representational power of restricted Boltzmann machines and deep belief networks. Neural computation, 20(6):1631–1649, 2008.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
P. Mehta, M. Bukov, C.-H. Wang, A. G. Day, C. Richardson, C. K. Fisher, and D. J. Schwab. A high-bias, low-variance introduction to machine learning for physicists. Physics reports, 810:1–124, 2019.
R. G. Melko, G. Carleo, J. Carrasquilla, and J. I. Cirac. Restricted boltzmann machines in quantum physics. Nature Physics, 15(9):887–892, 2019.
N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087–1092, 1953.
G. Montúfar. Restricted Boltzmann machines: Introduction and review. In Information Geometry and Its Applications IV, pages 75–115. Springer, 2016.
F. Ricci-Tersenghi. The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods. Journal of Statistical Mechanics: Theory and Experiment, 2012(08):P08015, 2012.
D. Sherrington and S. Kirkpatrick. Solvable model of a spin-glass. Physical review letters, 35(26):1792, 1975.
D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140–1144, 2018.
P. Smolensky. Information processing in dynamical systems: Foundations of harmony theory. In D. E. Rumelhart and J. L. McLelland, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations, chapter 6, pages 194–281. MIT Press, Cambridge, 1986.
T. Tieleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In Proceedings of the 25th international conference on Machine learning, pages 1064–1071, 2008.
J. Tubiana, S. Cocco, and R. Monasson. Learning protein constitutive motifs from sequence data. Elife, 8:e39397, 2019.
G. Van Rossum and F. L. Drake Jr. Python reference manual. Centrum voor Wiskunde en Informatica Amsterdam, 1995.
U. Wolff. Collective Monte Carlo updating for spin systems. Physical Review Letters, 62(4):361, 1989.
B. Yelmen, A. Decelle, L. Ongaro, D. Marnetto, C. Tallec, F. Montinaro, C. Furtlehner, L. Pagani, and F. Jay. Creating artificial human genomes using generative neural networks. PLoS genetics, 17(2):e1009303, 2021.
dc.rights.coar.fl_str_mv http://purl.org/coar/access_right/c_abf2
dc.rights.license.spa.fl_str_mv Atribución-NoComercial-SinDerivadas 4.0 Internacional
dc.rights.uri.spa.fl_str_mv http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.accessrights.spa.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv Atribución-NoComercial-SinDerivadas 4.0 Internacional
http://creativecommons.org/licenses/by-nc-nd/4.0/
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.extent.spa.fl_str_mv 39 páginas
dc.format.mimetype.spa.fl_str_mv application/pdf
dc.publisher.spa.fl_str_mv Universidad Nacional de Colombia
dc.publisher.program.spa.fl_str_mv Bogotá - Ciencias - Maestría en Ciencias - Física
dc.publisher.faculty.spa.fl_str_mv Facultad de Ciencias
dc.publisher.place.spa.fl_str_mv Bogotá, Colombia
dc.publisher.branch.spa.fl_str_mv Universidad Nacional de Colombia - Sede Bogotá
institution Universidad Nacional de Colombia
bitstream.url.fl_str_mv https://repositorio.unal.edu.co/bitstream/unal/83483/3/license.txt
https://repositorio.unal.edu.co/bitstream/unal/83483/4/1032483566.2022.pdf
https://repositorio.unal.edu.co/bitstream/unal/83483/5/1032483566.2022.pdf.jpg
bitstream.checksum.fl_str_mv eb34b1cf90b7e1103fc9dfd26be24b4a
adfc7620329e6974e301700b6637a434
33041cc3e0ec048df28bd7ce7753c738
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
MD5
repository.name.fl_str_mv Repositorio Institucional Universidad Nacional de Colombia
repository.mail.fl_str_mv repositorio_nal@unal.edu.co
_version_ 1814089337542803456
spelling Atribución-NoComercial-SinDerivadas 4.0 Internacionalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccesshttp://purl.org/coar/access_right/c_abf2Giraldo Gallo, José Jairo3b93634cccb8d7ef881dbabda8d23457Seoane Bartolomé, Beatriz7bfb1d0397926e8d8873d82b98231f28Navas Gómez, Alfonso de Jesús2dc6e675ade88867eb3efe3c7ec147312023-02-15T15:50:31Z2023-02-15T15:50:31Z2022-11-15https://repositorio.unal.edu.co/handle/unal/83483Universidad Nacional de ColombiaRepositorio Institucional Universidad Nacional de Colombiahttps://repositorio.unal.edu.co/ilustracionesAunque los métodos de inteligencia artificial basados en aprendizaje automatizado son considerados como una de las tecnologías disruptivas de nuestros tiempos, el entendimiento de estas herramientas yace muy por detrás de su éxito práctico. La física estadística de sistemas desordenados goza de una larga historia estudiando problemas de inferencia y aprendizaje usando sus propias herramientas. Siguiendo con esta tradición, en este trabajo final de maestría se estudió cómo el protocolo de aprendizaje afecta a los patrones extraídos por una Máquina Restringida de Boltzmann. En particular, se entrenaron máquinas dentro y fuera del equilibrio con muestras del modelo de Ising en 1 y 2 dimensiones para luego, usando un nuevo método de inferencia, extraer la matriz de acoplamientos del modelo efectivo aprendido en cada caso. Este experimento permitió dilucidar algunas consecuencias de los regímenes de entrenamiento dentro y fuera de equilibrio. Adicionalmente, se exploró el potencial del uso de las Máquinas Restringidas de Boltzmann para la extracción automática de patrones para muestras similares a las del modelo de Ising, siendo este el primer paso para abordar problemas más complejos. (Texto tomado de la fuente)Although machine learning based artificial intelligence is considered as one of the most disruptive technologies of our age, the understanding of many of these methods lies behind their practical success. Statistical physics of disordered systems has a long history studying inference problems and learning processes with its own tools, shedding light on the underlying mechanisms of many machine learning models. Following this tradition, in this master's thesis we studied how the training protocol affects the model and the features extracted by an unsupervised machine learning method called Restricted Boltzmann Machine. In particular, we trained machines in and out-of-equilibrium learning regimes with Ising Model samples and then, using a novel pattern extraction protocol developed in this work, we inferred the coupling matrix of the effective Ising model learned in each case. Such experiment allowed us to elucidate some consequences of equilibrium and non-equilibrium training regimes. Additionally, we explored the potential use of restricted Boltzmann machine as an inference tool for Ising model-like sample data, being the first step towards to tackle more complex problems.MaestríaMagíster en Ciencias - Física39 páginasapplication/pdfengUniversidad Nacional de ColombiaBogotá - Ciencias - Maestría en Ciencias - FísicaFacultad de CienciasBogotá, ColombiaUniversidad Nacional de Colombia - Sede BogotáExploring in and out-of-equilibrium learning regimes of restricted Boltzmann machinesExplorando los regímenes de aprendizaje dentro y fuera del equilibrio de las máquinas de Boltzmann restringidasTrabajo de grado - Maestríainfo:eu-repo/semantics/masterThesisinfo:eu-repo/semantics/acceptedVersionTexthttp://purl.org/redcol/resource_type/TMD. H. Ackley, G. E. Hinton, and T. J. Sejnowski. A learning algorithm for boltzmann machines. Cognitive science, 9(1):147–169, 1985.D. J. Amit and V. Martin-Mayor. Field theory, the renormalization group, and critical phenomena: graphs to computers. World Scientific Publishing Company, 2005.A. Bakk and J. S. Høye. One-dimensional ising model applied to protein folding. Physica A: Statistical Mechanics and its Applications, 323:504–518, 2003.H. Ballesteros and V. Martín-Mayor. Test for random number generators: Schwinger-Dyson equations for the ising model. Physical Review E, 58(5):6787, 1998.S. Behnel, R. Bradshaw, C. Citro, L. Dalcin, D. S. Seljebotn, and K. Smith. Cython: The best of both worlds. Computing in Science & Engineering, 13(2):31–39, 2011.N. Béreux, A. Decelle, C. Furtlehner, and B. Seoane. Learning a restricted Boltzmann machine using biased Monte Carlo sampling. arXiv preprint arXiv:2206.01310, 2022.C. M. Bishop. Pattern Recognition and Machine Learning. Springer, New York, 2006.B. Bravi, J. Tubiana, S. Cocco, R. Monasson, T. Mora, and A. M. Walczak. Rbm-mhc: A semi-supervised machine-learning method for sample-specific prediction of antigen presentation by hla-i alleles. Cell systems, 12(2):195–202, 2021.L. Brocchieri and S. Karlin. Protein length in eukaryotic and prokaryotic proteomes. Nucleic acids research, 33(10):3390–3400, 2005.G. Cossu, L. Del Debbio, T. Giani, A. Khamseh, and M. Wilson. Machine learning determination of dynamical parameters: The ising model case. Physical Review B, 100(6):064304, 2019.A. Decelle. TorchRBM. https://github.com/AurelienDecelle/TorchRBM, 2021. Accessed: 20-07-2022.A. Decelle and C. Furtlehner. Restricted Boltzmann machine: Recent advances and mean-field theory. Chinese Physics B, 30(4):040202, 2021.A. Decelle, C. Furtlehner, and B. Seoane. Equilibrium and non-equilibrium regimes in the learning of restricted boltzmann machines. arXiv preprint arXiv:2105.13889, 2021.A. Fischer and C. Igel. An introduction to restricted boltzmann machines. In Iberoamerican congress on pattern recognition, pages 14–36. Springer, 2012.M. Harsh, J. Tubiana, S. Cocco, and R. Monasson. ‘place-cell’ emergence and learning of invariant data with restricted boltzmann machines: breaking and dynamical restoration of continuous symmetries in the weight space. Journal of Physics A: Mathematical and Theoretical, 53(17):174002, 2020.W. K. Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika, 57(1):97–109, 1953.T. L. Hill. Generalization of the one-dimensional Ising model applicable to helix transitions in nucleic acids and proteins. The Journal of Chemical Physics, 30(2):383–387, 1959.G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):1771–1800, 2002.G. E. Hinton. A practical guide to training restricted Boltzmann machines. In Neural networks: Tricks of the trade, pages 599–619. Springer, 2012.G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504–507, 2006.R. D. Hjelm, V. D. Calhoun, R. Salakhutdinov, E. A. Allen, T. Adali, and S. M. Plis. Restricted boltzmann machines for neuroimaging: an application in identifying intrinsic networks. NeuroImage, 96:245–260, 2014.N. Le Roux and Y. Bengio. Representational power of restricted Boltzmann machines and deep belief networks. Neural computation, 20(6):1631–1649, 2008.Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.P. Mehta, M. Bukov, C.-H. Wang, A. G. Day, C. Richardson, C. K. Fisher, and D. J. Schwab. A high-bias, low-variance introduction to machine learning for physicists. Physics reports, 810:1–124, 2019.R. G. Melko, G. Carleo, J. Carrasquilla, and J. I. Cirac. Restricted boltzmann machines in quantum physics. Nature Physics, 15(9):887–892, 2019.N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller. Equation of state calculations by fast computing machines. The journal of chemical physics, 21(6):1087–1092, 1953.G. Montúfar. Restricted Boltzmann machines: Introduction and review. In Information Geometry and Its Applications IV, pages 75–115. Springer, 2016.F. Ricci-Tersenghi. The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods. Journal of Statistical Mechanics: Theory and Experiment, 2012(08):P08015, 2012.D. Sherrington and S. Kirkpatrick. Solvable model of a spin-glass. Physical review letters, 35(26):1792, 1975.D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140–1144, 2018.P. Smolensky. Information processing in dynamical systems: Foundations of harmony theory. In D. E. Rumelhart and J. L. McLelland, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations, chapter 6, pages 194–281. MIT Press, Cambridge, 1986.T. Tieleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In Proceedings of the 25th international conference on Machine learning, pages 1064–1071, 2008.J. Tubiana, S. Cocco, and R. Monasson. Learning protein constitutive motifs from sequence data. Elife, 8:e39397, 2019.G. Van Rossum and F. L. Drake Jr. Python reference manual. Centrum voor Wiskunde en Informatica Amsterdam, 1995.U. Wolff. Collective Monte Carlo updating for spin systems. Physical Review Letters, 62(4):361, 1989.B. Yelmen, A. Decelle, L. Ongaro, D. Marnetto, C. Tallec, F. Montinaro, C. Furtlehner, L. Pagani, and F. Jay. Creating artificial human genomes using generative neural networks. PLoS genetics, 17(2):e1009303, 2021.Complejidad computacionalSistemas expertosComputational complexityInteligencia artificialAprendizaje automatizadoFísica estadística de sistemas desordenadosMáquinas de Boltzmann restringidasMétodos de Monte-CarloArtificial intelligenceStatistical physics of disordered systemsRestricted Boltzmann machinesMonte-Carlo methodsMachine learningLICENSElicense.txtlicense.txttext/plain; charset=utf-85879https://repositorio.unal.edu.co/bitstream/unal/83483/3/license.txteb34b1cf90b7e1103fc9dfd26be24b4aMD53ORIGINAL1032483566.2022.pdf1032483566.2022.pdfTesis de Maestría en Ciencias - Físicaapplication/pdf912671https://repositorio.unal.edu.co/bitstream/unal/83483/4/1032483566.2022.pdfadfc7620329e6974e301700b6637a434MD54THUMBNAIL1032483566.2022.pdf.jpg1032483566.2022.pdf.jpgGenerated Thumbnailimage/jpeg4341https://repositorio.unal.edu.co/bitstream/unal/83483/5/1032483566.2022.pdf.jpg33041cc3e0ec048df28bd7ce7753c738MD55unal/83483oai:repositorio.unal.edu.co:unal/834832023-08-16 23:03:40.815Repositorio Institucional Universidad Nacional de Colombiarepositorio_nal@unal.edu.coUEFSVEUgMS4gVMOJUk1JTk9TIERFIExBIExJQ0VOQ0lBIFBBUkEgUFVCTElDQUNJw5NOIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KCkxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgbG9zIGRlcmVjaG9zIHBhdHJpbW9uaWFsZXMgZGUgYXV0b3IsIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEsIGxpbWl0YWRhIHkgZ3JhdHVpdGEgc29icmUgbGEgb2JyYSBxdWUgc2UgaW50ZWdyYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsLCBiYWpvIGxvcyBzaWd1aWVudGVzIHTDqXJtaW5vczoKCgphKQlMb3MgYXV0b3JlcyB5L28gbG9zIHRpdHVsYXJlcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBzb2JyZSBsYSBvYnJhIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEgcGFyYSByZWFsaXphciBsb3Mgc2lndWllbnRlcyBhY3RvcyBzb2JyZSBsYSBvYnJhOiBpKSByZXByb2R1Y2lyIGxhIG9icmEgZGUgbWFuZXJhIGRpZ2l0YWwsIHBlcm1hbmVudGUgbyB0ZW1wb3JhbCwgaW5jbHV5ZW5kbyBlbCBhbG1hY2VuYW1pZW50byBlbGVjdHLDs25pY28sIGFzw60gY29tbyBjb252ZXJ0aXIgZWwgZG9jdW1lbnRvIGVuIGVsIGN1YWwgc2UgZW5jdWVudHJhIGNvbnRlbmlkYSBsYSBvYnJhIGEgY3VhbHF1aWVyIG1lZGlvIG8gZm9ybWF0byBleGlzdGVudGUgYSBsYSBmZWNoYSBkZSBsYSBzdXNjcmlwY2nDs24gZGUgbGEgcHJlc2VudGUgbGljZW5jaWEsIHkgaWkpIGNvbXVuaWNhciBhbCBww7pibGljbyBsYSBvYnJhIHBvciBjdWFscXVpZXIgbWVkaW8gbyBwcm9jZWRpbWllbnRvLCBlbiBtZWRpb3MgYWzDoW1icmljb3MgbyBpbmFsw6FtYnJpY29zLCBpbmNsdXllbmRvIGxhIHB1ZXN0YSBhIGRpc3Bvc2ljacOzbiBlbiBhY2Nlc28gYWJpZXJ0by4gQWRpY2lvbmFsIGEgbG8gYW50ZXJpb3IsIGVsIGF1dG9yIHkvbyB0aXR1bGFyIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgcGFyYSBxdWUsIGVuIGxhIHJlcHJvZHVjY2nDs24geSBjb211bmljYWNpw7NuIGFsIHDDumJsaWNvIHF1ZSBsYSBVbml2ZXJzaWRhZCByZWFsaWNlIHNvYnJlIGxhIG9icmEsIGhhZ2EgbWVuY2nDs24gZGUgbWFuZXJhIGV4cHJlc2EgYWwgdGlwbyBkZSBsaWNlbmNpYSBDcmVhdGl2ZSBDb21tb25zIGJham8gbGEgY3VhbCBlbCBhdXRvciB5L28gdGl0dWxhciBkZXNlYSBvZnJlY2VyIHN1IG9icmEgYSBsb3MgdGVyY2Vyb3MgcXVlIGFjY2VkYW4gYSBkaWNoYSBvYnJhIGEgdHJhdsOpcyBkZWwgUmVwb3NpdG9yaW8gSW5zdGl0dWNpb25hbCwgY3VhbmRvIHNlYSBlbCBjYXNvLiBFbCBhdXRvciB5L28gdGl0dWxhciBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBwb2Ryw6EgZGFyIHBvciB0ZXJtaW5hZGEgbGEgcHJlc2VudGUgbGljZW5jaWEgbWVkaWFudGUgc29saWNpdHVkIGVsZXZhZGEgYSBsYSBEaXJlY2Npw7NuIE5hY2lvbmFsIGRlIEJpYmxpb3RlY2FzIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLiAKCmIpIAlMb3MgYXV0b3JlcyB5L28gdGl0dWxhcmVzIGRlIGxvcyBkZXJlY2hvcyBwYXRyaW1vbmlhbGVzIGRlIGF1dG9yIHNvYnJlIGxhIG9icmEgY29uZmllcmVuIGxhIGxpY2VuY2lhIHNlw7FhbGFkYSBlbiBlbCBsaXRlcmFsIGEpIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8gcG9yIGVsIHRpZW1wbyBkZSBwcm90ZWNjacOzbiBkZSBsYSBvYnJhIGVuIHRvZG9zIGxvcyBwYcOtc2VzIGRlbCBtdW5kbywgZXN0byBlcywgc2luIGxpbWl0YWNpw7NuIHRlcnJpdG9yaWFsIGFsZ3VuYS4KCmMpCUxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBtYW5pZmllc3RhbiBlc3RhciBkZSBhY3VlcmRvIGNvbiBxdWUgbGEgcHJlc2VudGUgbGljZW5jaWEgc2Ugb3RvcmdhIGEgdMOtdHVsbyBncmF0dWl0bywgcG9yIGxvIHRhbnRvLCByZW51bmNpYW4gYSByZWNpYmlyIGN1YWxxdWllciByZXRyaWJ1Y2nDs24gZWNvbsOzbWljYSBvIGVtb2x1bWVudG8gYWxndW5vIHBvciBsYSBwdWJsaWNhY2nDs24sIGRpc3RyaWJ1Y2nDs24sIGNvbXVuaWNhY2nDs24gcMO6YmxpY2EgeSBjdWFscXVpZXIgb3RybyB1c28gcXVlIHNlIGhhZ2EgZW4gbG9zIHTDqXJtaW5vcyBkZSBsYSBwcmVzZW50ZSBsaWNlbmNpYSB5IGRlIGxhIGxpY2VuY2lhIENyZWF0aXZlIENvbW1vbnMgY29uIHF1ZSBzZSBwdWJsaWNhLgoKZCkJUXVpZW5lcyBmaXJtYW4gZWwgcHJlc2VudGUgZG9jdW1lbnRvIGRlY2xhcmFuIHF1ZSBwYXJhIGxhIGNyZWFjacOzbiBkZSBsYSBvYnJhLCBubyBzZSBoYW4gdnVsbmVyYWRvIGxvcyBkZXJlY2hvcyBkZSBwcm9waWVkYWQgaW50ZWxlY3R1YWwsIGluZHVzdHJpYWwsIG1vcmFsZXMgeSBwYXRyaW1vbmlhbGVzIGRlIHRlcmNlcm9zLiBEZSBvdHJhIHBhcnRlLCAgcmVjb25vY2VuIHF1ZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhY3TDumEgY29tbyB1biB0ZXJjZXJvIGRlIGJ1ZW5hIGZlIHkgc2UgZW5jdWVudHJhIGV4ZW50YSBkZSBjdWxwYSBlbiBjYXNvIGRlIHByZXNlbnRhcnNlIGFsZ8O6biB0aXBvIGRlIHJlY2xhbWFjacOzbiBlbiBtYXRlcmlhIGRlIGRlcmVjaG9zIGRlIGF1dG9yIG8gcHJvcGllZGFkIGludGVsZWN0dWFsIGVuIGdlbmVyYWwuIFBvciBsbyB0YW50bywgbG9zIGZpcm1hbnRlcyAgYWNlcHRhbiBxdWUgY29tbyB0aXR1bGFyZXMgw7puaWNvcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciwgYXN1bWlyw6FuIHRvZGEgbGEgcmVzcG9uc2FiaWxpZGFkIGNpdmlsLCBhZG1pbmlzdHJhdGl2YSB5L28gcGVuYWwgcXVlIHB1ZWRhIGRlcml2YXJzZSBkZSBsYSBwdWJsaWNhY2nDs24gZGUgbGEgb2JyYS4gIAoKZikJQXV0b3JpemFuIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgaW5jbHVpciBsYSBvYnJhIGVuIGxvcyBhZ3JlZ2Fkb3JlcyBkZSBjb250ZW5pZG9zLCBidXNjYWRvcmVzIGFjYWTDqW1pY29zLCBtZXRhYnVzY2Fkb3Jlcywgw61uZGljZXMgeSBkZW3DoXMgbWVkaW9zIHF1ZSBzZSBlc3RpbWVuIG5lY2VzYXJpb3MgcGFyYSBwcm9tb3ZlciBlbCBhY2Nlc28geSBjb25zdWx0YSBkZSBsYSBtaXNtYS4gCgpnKQlFbiBlbCBjYXNvIGRlIGxhcyB0ZXNpcyBjcmVhZGFzIHBhcmEgb3B0YXIgZG9ibGUgdGl0dWxhY2nDs24sIGxvcyBmaXJtYW50ZXMgc2Vyw6FuIGxvcyByZXNwb25zYWJsZXMgZGUgY29tdW5pY2FyIGEgbGFzIGluc3RpdHVjaW9uZXMgbmFjaW9uYWxlcyBvIGV4dHJhbmplcmFzIGVuIGNvbnZlbmlvLCBsYXMgbGljZW5jaWFzIGRlIGFjY2VzbyBhYmllcnRvIENyZWF0aXZlIENvbW1vbnMgeSBhdXRvcml6YWNpb25lcyBhc2lnbmFkYXMgYSBzdSBvYnJhIHBhcmEgbGEgcHVibGljYWNpw7NuIGVuIGVsIFJlcG9zaXRvcmlvIEluc3RpdHVjaW9uYWwgVU5BTCBkZSBhY3VlcmRvIGNvbiBsYXMgZGlyZWN0cmljZXMgZGUgbGEgUG9sw610aWNhIEdlbmVyYWwgZGUgbGEgQmlibGlvdGVjYSBEaWdpdGFsLgoKCmgpCVNlIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgY29tbyByZXNwb25zYWJsZSBkZWwgdHJhdGFtaWVudG8gZGUgZGF0b3MgcGVyc29uYWxlcywgZGUgYWN1ZXJkbyBjb24gbGEgbGV5IDE1ODEgZGUgMjAxMiBlbnRlbmRpZW5kbyBxdWUgc2UgZW5jdWVudHJhbiBiYWpvIG1lZGlkYXMgcXVlIGdhcmFudGl6YW4gbGEgc2VndXJpZGFkLCBjb25maWRlbmNpYWxpZGFkIGUgaW50ZWdyaWRhZCwgeSBzdSB0cmF0YW1pZW50byB0aWVuZSB1bmEgZmluYWxpZGFkIGhpc3TDs3JpY2EsIGVzdGFkw61zdGljYSBvIGNpZW50w61maWNhIHNlZ8O6biBsbyBkaXNwdWVzdG8gZW4gbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMuCgoKClBBUlRFIDIuIEFVVE9SSVpBQ0nDk04gUEFSQSBQVUJMSUNBUiBZIFBFUk1JVElSIExBIENPTlNVTFRBIFkgVVNPIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KClNlIGF1dG9yaXphIGxhIHB1YmxpY2FjacOzbiBlbGVjdHLDs25pY2EsIGNvbnN1bHRhIHkgdXNvIGRlIGxhIG9icmEgcG9yIHBhcnRlIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgZGUgc3VzIHVzdWFyaW9zIGRlIGxhIHNpZ3VpZW50ZSBtYW5lcmE6CgphLglDb25jZWRvIGxpY2VuY2lhIGVuIGxvcyB0w6lybWlub3Mgc2XDsWFsYWRvcyBlbiBsYSBwYXJ0ZSAxIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8sIGNvbiBlbCBvYmpldGl2byBkZSBxdWUgbGEgb2JyYSBlbnRyZWdhZGEgc2VhIHB1YmxpY2FkYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgcHVlc3RhIGEgZGlzcG9zaWNpw7NuIGVuIGFjY2VzbyBhYmllcnRvIHBhcmEgc3UgY29uc3VsdGEgcG9yIGxvcyB1c3VhcmlvcyBkZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSAgYSB0cmF2w6lzIGRlIGludGVybmV0LgoKCgpQQVJURSAzIEFVVE9SSVpBQ0nDk04gREUgVFJBVEFNSUVOVE8gREUgREFUT1MgUEVSU09OQUxFUy4KCkxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLCBjb21vIHJlc3BvbnNhYmxlIGRlbCBUcmF0YW1pZW50byBkZSBEYXRvcyBQZXJzb25hbGVzLCBpbmZvcm1hIHF1ZSBsb3MgZGF0b3MgZGUgY2Fyw6FjdGVyIHBlcnNvbmFsIHJlY29sZWN0YWRvcyBtZWRpYW50ZSBlc3RlIGZvcm11bGFyaW8sIHNlIGVuY3VlbnRyYW4gYmFqbyBtZWRpZGFzIHF1ZSBnYXJhbnRpemFuIGxhIHNlZ3VyaWRhZCwgY29uZmlkZW5jaWFsaWRhZCBlIGludGVncmlkYWQgeSBzdSB0cmF0YW1pZW50byBzZSByZWFsaXphIGRlIGFjdWVyZG8gYWwgY3VtcGxpbWllbnRvIG5vcm1hdGl2byBkZSBsYSBMZXkgMTU4MSBkZSAyMDEyIHkgZGUgbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMgZGUgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEuIFB1ZWRlIGVqZXJjZXIgc3VzIGRlcmVjaG9zIGNvbW8gdGl0dWxhciBhIGNvbm9jZXIsIGFjdHVhbGl6YXIsIHJlY3RpZmljYXIgeSByZXZvY2FyIGxhcyBhdXRvcml6YWNpb25lcyBkYWRhcyBhIGxhcyBmaW5hbGlkYWRlcyBhcGxpY2FibGVzIGEgdHJhdsOpcyBkZSBsb3MgY2FuYWxlcyBkaXNwdWVzdG9zIHkgZGlzcG9uaWJsZXMgZW4gd3d3LnVuYWwuZWR1LmNvIG8gZS1tYWlsOiBwcm90ZWNkYXRvc19uYUB1bmFsLmVkdS5jbyIKClRlbmllbmRvIGVuIGN1ZW50YSBsbyBhbnRlcmlvciwgYXV0b3Jpem8gZGUgbWFuZXJhIHZvbHVudGFyaWEsIHByZXZpYSwgZXhwbMOtY2l0YSwgaW5mb3JtYWRhIGUgaW5lcXXDrXZvY2EgYSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhIHRyYXRhciBsb3MgZGF0b3MgcGVyc29uYWxlcyBkZSBhY3VlcmRvIGNvbiBsYXMgZmluYWxpZGFkZXMgZXNwZWPDrWZpY2FzIHBhcmEgZWwgZGVzYXJyb2xsbyB5IGVqZXJjaWNpbyBkZSBsYXMgZnVuY2lvbmVzIG1pc2lvbmFsZXMgZGUgZG9jZW5jaWEsIGludmVzdGlnYWNpw7NuIHkgZXh0ZW5zacOzbiwgYXPDrSBjb21vIGxhcyByZWxhY2lvbmVzIGFjYWTDqW1pY2FzLCBsYWJvcmFsZXMsIGNvbnRyYWN0dWFsZXMgeSB0b2RhcyBsYXMgZGVtw6FzIHJlbGFjaW9uYWRhcyBjb24gZWwgb2JqZXRvIHNvY2lhbCBkZSBsYSBVbml2ZXJzaWRhZC4gCgo=