Overcoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion Problems
ilustraciones, diagramas, fotografías
- Autores:
-
Yanguas Rojas, David Reinerio
- Tipo de recurso:
- Doctoral thesis
- Fecha de publicación:
- 2023
- Institución:
- Universidad Nacional de Colombia
- Repositorio:
- Universidad Nacional de Colombia
- Idioma:
- eng
- OAI Identifier:
- oai:repositorio.unal.edu.co:unal/85427
- Palabra clave:
- 620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingeniería
Automation
Robótica
Robotics
Automatización
Algoritmos
Algorithms
Reinforcement learning
Humanoid Robotics
Imitation Learning
Non-Linear Control
Robot Training
Bipedal Locomotion
Humanoid Locomotion
Artificial Learning Techniques
Reality Gap
Sim to Real
- Rights
- openAccess
- License
- Reconocimiento 4.0 Internacional
| id |
UNACIONAL2_625c08e1ea3db30bb81c53fed2740c36 |
|---|---|
| oai_identifier_str |
oai:repositorio.unal.edu.co:unal/85427 |
| network_acronym_str |
UNACIONAL2 |
| network_name_str |
Universidad Nacional de Colombia |
| repository_id_str |
|
| dc.title.eng.fl_str_mv |
Overcoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion Problems |
| dc.title.translated.spa.fl_str_mv |
Superando la brecha de la realidad: Algoritmos de aprendizaje por imitación y por refuerzos para problemas de locomoción robótica bípeda |
| title |
Overcoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion Problems |
| spellingShingle |
Overcoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion Problems 620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingeniería Automation Robótica Robotics Automatización Algoritmos Algorithms Reinforcement learning Humanoid Robotics Imitation Learning Non-Linear Control Robot Training Bipedal Locomotion Humanoid Locomotion Artificial Learning Techniques Reality Gap Sim to Real |
| title_short |
Overcoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion Problems |
| title_full |
Overcoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion Problems |
| title_fullStr |
Overcoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion Problems |
| title_full_unstemmed |
Overcoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion Problems |
| title_sort |
Overcoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion Problems |
| dc.creator.fl_str_mv |
Yanguas Rojas, David Reinerio |
| dc.contributor.advisor.none.fl_str_mv |
Mojica Nava, Eduardo Alirio |
| dc.contributor.author.none.fl_str_mv |
Yanguas Rojas, David Reinerio |
| dc.contributor.researchgroup.spa.fl_str_mv |
Programa de Investigacion sobre Adquisicion y Analisis de Señales Paas-Un |
| dc.contributor.orcid.spa.fl_str_mv |
0000-0001-5874-721X |
| dc.contributor.cvlac.spa.fl_str_mv |
Yanguas Rojas, David |
| dc.contributor.researchgate.spa.fl_str_mv |
David R. Yanguas Rojas |
| dc.contributor.googlescholar.spa.fl_str_mv |
David Yanguas-Rojas |
| dc.subject.ddc.spa.fl_str_mv |
620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingeniería |
| topic |
620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingeniería Automation Robótica Robotics Automatización Algoritmos Algorithms Reinforcement learning Humanoid Robotics Imitation Learning Non-Linear Control Robot Training Bipedal Locomotion Humanoid Locomotion Artificial Learning Techniques Reality Gap Sim to Real |
| dc.subject.armarc.none.fl_str_mv |
Automation |
| dc.subject.lemb.none.fl_str_mv |
Robótica Robotics Automatización Algoritmos Algorithms |
| dc.subject.proposal.eng.fl_str_mv |
Reinforcement learning Humanoid Robotics Imitation Learning Non-Linear Control Robot Training Bipedal Locomotion Humanoid Locomotion Artificial Learning Techniques Reality Gap Sim to Real |
| description |
ilustraciones, diagramas, fotografías |
| publishDate |
2023 |
| dc.date.issued.none.fl_str_mv |
2023 |
| dc.date.accessioned.none.fl_str_mv |
2024-01-24T20:37:35Z |
| dc.date.available.none.fl_str_mv |
2024-01-24T20:37:35Z |
| dc.type.spa.fl_str_mv |
Trabajo de grado - Doctorado |
| dc.type.driver.spa.fl_str_mv |
info:eu-repo/semantics/doctoralThesis |
| dc.type.version.spa.fl_str_mv |
info:eu-repo/semantics/acceptedVersion |
| dc.type.coar.spa.fl_str_mv |
http://purl.org/coar/resource_type/c_db06 |
| dc.type.content.spa.fl_str_mv |
Audiovisual Text |
| dc.type.redcol.spa.fl_str_mv |
http://purl.org/redcol/resource_type/TD |
| format |
http://purl.org/coar/resource_type/c_db06 |
| status_str |
acceptedVersion |
| dc.identifier.uri.none.fl_str_mv |
https://repositorio.unal.edu.co/handle/unal/85427 |
| dc.identifier.instname.spa.fl_str_mv |
Universidad Nacional de Colombia |
| dc.identifier.reponame.spa.fl_str_mv |
Repositorio Institucional Universidad Nacional de Colombia |
| dc.identifier.repourl.spa.fl_str_mv |
https://repositorio.unal.edu.co/ |
| url |
https://repositorio.unal.edu.co/handle/unal/85427 https://repositorio.unal.edu.co/ |
| identifier_str_mv |
Universidad Nacional de Colombia Repositorio Institucional Universidad Nacional de Colombia |
| dc.language.iso.spa.fl_str_mv |
eng |
| language |
eng |
| dc.relation.references.spa.fl_str_mv |
[Arcos-Legarda et al., 2019] Arcos-Legarda, J., Cortes-Romero, J., Beltran-Pulido, A., and Tovar, A. (2019). Hybrid disturbance rejection control of dynamic bipedal robots. Multibody System Dynamics, 46:281–306. [Belongie, 2023] Belongie, S. (2023). Rodrigues’ rotation formula. Created by Eric W. Weisstein. [Burda et al., 2018] Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., and Efros, A. A. (2018). Large-Scale Study of Curiosity-Driven Learning. ArXiv. [Carnegie Mellon University Graphics Lab, 2004] Carnegie Mellon University Graphics Lab (2004). CMU Motion Capture Database. The database was created with funding from NSF EIA-0196217. [Chebotar et al., 2018] Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., and Fox, D. (2018). Closing the sim-to-real loop: Adapting simulation randomization with real world experience. [Chernova and Thomaz, 2014] Chernova, S. and Thomaz, A. L. (2014). Robot Learning from Human Teachers. Synthesis Lectures on Artificial Intelligence and Machine Learning, 8(3):1–121. [Chernova and Veloso, 2010] Chernova, S. and Veloso, M. (2010). Confidence-based multirobot learning from demonstration. International Journal of Social Robotics, 2(2):195–215. [Chevallereau et al., 2014] Chevallereau, C., Sinnet, R. W., Ames, A. D., and Universit´e, L. (2014). Models feedback control and open problems of 3d bipedal robotic walking. Automatica. [Collins et al., 2005] Collins, S., Ruina, A., Tedrake, R., and Wisse, M. (2005). Efficient bipedal robots based on passive-dynamic walkers. Science, 307:1082–1085. [Ding et al., 2019] Ding, J., Zhou, C., and Xiao, X. (2019). Energy-Efficient Bipedal Gait Pattern Generation via CoM Acceleration Optimization. IEEE-RAS International Conference on Humanoid Robots, 2018-Novem:238–244. [Duan et al., 2017] Duan, Y., Andrychowicz, M., Stadie, B. C., Ho, J., Schneider, J., Sutskever, I., Abbeel, P., and Zaremba, W. (2017). One-Shot Imitation Learning. ArXiv. [Duan et al., 2016] Duan, Y., Chen, X., Houthooft, R., Schulman, J., and Abbeel, P. (2016). Benchmarking Deep Reinforcement Learning for Continuous Control. ArXiv. [Erez et al., 2015] Erez, T., Lowrey, K., Tassa, Y., Kumar, V., Kolev, S., and Todorov, E. (2015). An integrated system for real-time model predictive control of humanoid robots. Proceedings on IEEE-RAS International Conference on Humanoid Robots, 2015- Febru(February):292–299. [García et al., 1989] Garc´ıa, C. E., Prett, D. M., and Morari, M. (1989). Model predictive control: Theory and practice—a survey. Automatica, 25(3):335–348. [Google, 2015] Google (2015). AlphaGo — DeepMind. https://deepmind.com/research/ alphago/. accessed 20-May-2019. [Heess et al., 2017] Heess, N., TB, D., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Eslami, S. M. A., Riedmiller, M., and Silver, D. (2017). Emergence of locomotion behaviours in rich environments. [Ijspeert, 2008] Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: A review. Neural Networks, 21(4):642–653. [Jumper et al., 2021] Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., ˇ Z´ıdek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Petersen, S., Reiman, D., Clancy, E., Zielinski, M., Steinegger, M., Pacholska, M., Berghammer, T., Bodenstein, S., Silver, D., Vinyals, O., Senior, A. W., Kavukcuoglu, K., Kohli, P., and Hassabis, D. (2021). Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583 – 589. Cited by: 8028; All Open Access, Green Open Access, Hybrid Gold Open Access. [Kingma and Ba, 2015] Kingma, D. P. and Ba, J. L. (2015). Adam: A method for stochastic optimization. [Kobayashi et al., 2018] Kobayashi, T., Sekiyama, K., Hasegawa, Y., Aoyama, T., and Fukuda, T. (2018). Virtual-dynamics-based reference gait speed generator for limit-cycle-based bipedal gait. ROBOMECH Journal, 5(1). [Koenemann et al., 2014] Koenemann, J., Burget, F., and Bennewitz, M. (2014). Real-time imitation of human whole-body motions by humanoids. Proceedings - IEEE International Conference on Robotics and Automation, pages 2806–2812. [Koos et al., 2013] Koos, S., Mouret, J. B., and Doncieux, S. (2013). The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Transactions on Evolutionary Computation, 17(1):122–145. [Kuindersma et al., 2016] Kuindersma, S., Deits, R., Fallon, M., Valenzuela, A., Dai, H., Permenter, F., Koolen, T., Marion, P., and Tedrake, R. (2016). Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot. Autonomous Robots, 40:429–455. [Lee et al., 2019] Lee, K., Kim, S., Lim, S., Choi, S., and Oh, S. (2019). Tsallis Reinforcement Learning: A Unified Framework for Maximum Entropy Reinforcement Learning. ArXiv. [Lillicrap et al., 2015] Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. ArXiv. [Lillicrap et al., 2019] Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2019). Continuous control with deep reinforcement learning. [Liu et al., 2022] Liu, S., Lever, G., Wang, Z., Merel, J., Eslami, S., Hennes, D., Czarnecki, W., Tassa, Y., Omidshafiei, S., Abdolmaleki, A., Graepel, T., and Heess, N. (2022). From motor control to team play in simulated humanoid football. Science Robotics, 7. [Loudon et al., 2008] Loudon, J. K. J. K., Swift, M., and Bell, S. (2008). The clinical orthopedic assessment guide. Human Kinetics. [Luenberger and Ye, 2021] Luenberger, D. G. and Ye, Y. (2021). Penalty and barrier methods. International Series in Operations Research and Management Science, 228. [Ma et al., 2021] Ma, L.-K., Yang, Z., Xin, T., Guo, B., and Yin, K. (2021). Learning and exploring motor skills with spacetime bounds. Computer Graphics Forum, 40(2):251–263. [Ma et al., 2018] Ma, W.-L., Or, Y., and Ames, A. D. (2018). Dynamic Walking on Slippery Surfaces: Demonstrating Stable Bipedal Gaits with Planned Ground Slippage. Proceedings on 2019 International Conference on Robotics and Automation (ICRA). [Mania et al., 2018] Mania, H., Guy, A., and Recht, B. (2018). Simple random search provides a competitive approach to reinforcement learning. Advances in Neural Information Processing Systems 31 (NeurIPS 2018). [Mark W. Spong, 2020] Mark W. Spong, Seth Hutchinson, M. V. (2020). Robot modeling and control. JOHN WILEY and SONS, INC. [Merel et al., 2017] Merel, J., Tassa, Y., TB, D., Srinivasan, S., Lemmon, J., Wang, Z., Wayne, G., and Heess, N. (2017). Learning human behaviors from motion capture by adversarial imitation. eprint arXiv:1707.02201. [Mnih et al., 2016] Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., and Kavukcuoglu, K. (2016). Asynchronous Methods for Deep Reinforcement Learning. [Mnih et al., 2015] Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540):529 – 533. Cited by: 15639. [Müller et al., 2022] Müller, M., Jazdi, N., and Weyrich, M. (2022). Self-improving models for the intelligent digital twin: Towards closing the reality-to-simulation gap. IFACPapersOnLine, 55:126–131. Nehaniv and Dautenhahn, 2002] Nehaniv, C. L. and Dautenhahn, K. (2002). The Correspondence Problem, page 41–61. MIT Press, Cambridge, MA, USA. [Nguyen and La, 2019] Nguyen, H. and La, H. (2019). Review of Deep Reinforcement Learning for Robot Manipulation. Proceedings - 3rd IEEE International Conference on Robotic Computing, IRC 2019, pages 590–595. [Niu et al., 2022] Niu, J., Hu, Y., Li, W., Huang, G., Han, Y., and Li, X. (2022). Closing the dynamics gap via adversarial and reinforcement learning for high-speed racing. Proceedings of the International Joint Conference on Neural Networks, 2022-July. [OpenAI, 2018] OpenAI (2018). Openai five. https://blog.openai.com/openai-five/. accessed 20-May-2019. [Recht, 2018] Recht, B. (2018). A Tour of Reinforcement Learning: The View from Continuous Control. eprint arXiv:1806.09460. [Robbins and Monro, 1951] Robbins, H. and Monro, S. (1951). A Stochastic Approximation Method. The Annals of Mathematical Statistics, 22(3):400 – 407. [ROBOTIS, 2022a] ROBOTIS (2022a). Robotis engineer kit 1 e-manual (visited on 2022/04/01). [ROBOTIS, 2022b] ROBOTIS (2022b). Robotis Engineer Kit 2 E-Manual, (visited on 2022/04/01). [ROBOTIS, 2022c] ROBOTIS (2022c). Robotis mini e-manual (visited on 2022/04/01). [ROBOTIS, 2022d] ROBOTIS (2022d). Robotis xl-320 e-manual (visited on 2022/04/01). [Rodriguez et al., 2018] Rodriguez, D., Brandenburger, A., and Behnke, S. (2018). Combining Simulations and Real-robot Experiments for Bayesian Optimization of Bipedal Gait Stabilization. Lecture Notes in Computer Science book series. [Rosolia and Borrelli, 2016] Rosolia, U. and Borrelli, F. (2016). Learning Model Predictive Control for Iterative Tasks. eprint arXiv:1609.01387, 4. [Salvato et al., 2021] Salvato, E., Fenu, G., Medvet, E., and Pellegrino, F. A. (2021). Crossing the reality gap: A survey on sim-to-real transferability of robot controllers in reinforcement learning. IEEE Access, 9:153171–153187. [Schulman et al., 2015] Schulman, J., Levine, S., Moritz, P., Jordan, M. I., and Abbeel, P. (2015). Trust Region Policy Optimization. Proceedings of the 32nd International Conference on Machine Learning. [Siciliano and Khatib, 2016] Siciliano, B. and Khatib, O. (2016). Springer handbook of robotics. Springer eBooks. [Siegwart et al., 2011] Siegwart, R., Nourbakhsh, I. R., and Scaramuzza, D. (2011). Introduction to autonomous mobile robots. Intelligent robotics and autonomous agents. The MIT Press/Massachusetts Institute of Technology. [Simba et al., 2016] Simba, K. R., Uchiyama, N., and Sano, S. (2016). Real-time smooth trajectory generation for nonholonomic mobile robots using b´ezier curves. Robotics and Computer-Integrated Manufacturing, 41. [Singh et al., 2019] Singh, A., Yang, L., Hartikainen, K., Finn, C., and Levine, S. (2019). End-to-End Robotic Reinforcement Learning without Reward Engineering. Robotics: Science and Systems. [Sutton and Barto, 2008] Sutton, R. S. and Barto, A. G. (2008). Reinforcement Learning. The MIT Press, London, England, second edi edition. [´Swiechowski et al., 2022] ´Swiechowski, M., Godlewski, K., Sawicki, B., and Ma´ndziuk, J. (2022). Monte carlo tree search: a review of recent modifications and applications. Artificial Intelligence Review, 56(3):2497–2562. [Tan et al., 2018] Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., and Vanhoucke, V. (2018). Sim-to-Real: Learning Agile Locomotion For Quadruped Robots. eprint arXiv:1804.10332. [Tedrake, 2022] Tedrake, R. (2022). Underactuated Robotics. MIT. [Tevatia and Schaal, 2000] Tevatia, G. and Schaal, S. (2000). Inverse kinematics for humanoid robots. Proceedings-IEEE International Conference on Robotics and Automation, 1(April):294–299. [Thorp, 2023] Thorp, H. H. (2023). Chatgpt is fun, but not an author. Science, 379(6630):313; All Open Access, Bronze Open Access. [Tobin et al., 2017] Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. (2017). Domain randomization for transferring deep neural networks from simulation to the real world. IEEE International Conference on Intelligent Robots and Systems, 2017-Septe:23–30. [Todorov, 2019] Todorov, E. (2019). Mujoco: Modeling, simulation and visualization of multi-joint dynamics with contact. http://www.mujoco.org/book/index.html. accessed 20-May-2019. [Uchibe, 2018] Uchibe, E. (2018). Cooperative and competitive reinforcement and imitation learning for a mixture of heterogeneous learning modules. Frontiers in Neurorobotics, 12(SEP). [Vinyals et al., 2019] Vinyals, O., Babuschkin, I., Chung, J., Mathieu, M., Jaderberg, M., Czarnecki, W. M., Dudzik, A., Huang, A., Georgiev, P., Powell, R., Ewalds, T., Horgan, D., Kroiss, M., Danihelka, I., Agapiou, J., Oh, J., Dalibard, V., Choi, D., Sifre, L., Sulsky, Y., Vezhnevets, S., Molloy, J., Cai, T., Budden, D., Paine, T., Gulcehre, C., Wang, Z., Pfaff, T., Pohlen, T., Wu, Y., Yogatama, D., Cohen, J., McKinney, K., Smith, O., Schaul, T., Lillicrap, T., Apps, C., Kavukcuoglu, K., Hassabis, D., and Silver, D. (2019). AlphaStar: Mastering the Real-Time Strategy Game StarCraft II. https://deepmind.com/ blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/{}. accessed 20-May-2019. [Vukobratovic and Borovac, 2004] Vukobratovic, M. and Borovac, B. (2004). Zero-moment point — thirty five years of its life. International Journal of Humanoid Robotics, 01:157– 173. [Vukobratovic and Juricic, 1969] Vukobratovic, M. and Juricic, D. (1969). Contribution to the synthesis of biped gait. IEEE Transactions on Biomedical Engineering, BME-16(1):1– 6. [Wampler, 1986] Wampler, C. W. (1986). Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods. IEEE Transactions on Systems, Man and Cybernetics, 16. [Watkins and Dayan, 1992] Watkins, C. J. C. H. and Dayan, P. (1992). Q-learning. Machine Learning, 8(3):279–292. [Westervelt et al., 2003] Westervelt, E. R., Grizzle, J. W., and Koditschek, D. E. (2003). Hybrid zero dynamics of planar biped walkers. IEEE Transactions on Automatic Control, 48:42–56. [Xie et al., 2019] Xie, Z., Clary, P., Dao, J., Morais, P., Hurst, J., and van de Panne, M. (2019). Iterative reinforcement learning based design of dynamic locomotion skills for cassie. 3rd Conference on Robot Learning (CoRL 2019), Osaka, Japan. [Xie et al., 2020] Xie, Z., Ling, H. Y., Kim, N. H., and Panne, M. V. D. (2020). Allsteps: Curriculum-driven learning of stepping stone skills. ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA 2020, pages 213–224. [Zou et al., 2019] Zou, F., Shen, L., Jie, Z., Zhang, W., and Liu, W. (2019). A sufficient condition for convergences of adam and rmsprop. volume 2019-June. |
| dc.rights.coar.fl_str_mv |
http://purl.org/coar/access_right/c_abf2 |
| dc.rights.license.spa.fl_str_mv |
Reconocimiento 4.0 Internacional |
| dc.rights.uri.spa.fl_str_mv |
http://creativecommons.org/licenses/by/4.0/ |
| dc.rights.accessrights.spa.fl_str_mv |
info:eu-repo/semantics/openAccess |
| rights_invalid_str_mv |
Reconocimiento 4.0 Internacional http://creativecommons.org/licenses/by/4.0/ http://purl.org/coar/access_right/c_abf2 |
| eu_rights_str_mv |
openAccess |
| dc.format.extent.spa.fl_str_mv |
xxi, 158 páginas |
| dc.format.mimetype.spa.fl_str_mv |
application/pdf |
| dc.publisher.spa.fl_str_mv |
Universidad Nacional de Colombia |
| dc.publisher.program.spa.fl_str_mv |
Bogotá - Ingeniería - Doctorado en Ingeniería - Ingeniería Mecánica y Mecatrónica |
| dc.publisher.faculty.spa.fl_str_mv |
Facultad de Ingeniería |
| dc.publisher.place.spa.fl_str_mv |
Bogotá, Colombia |
| dc.publisher.branch.spa.fl_str_mv |
Universidad Nacional de Colombia - Sede Bogotá |
| institution |
Universidad Nacional de Colombia |
| bitstream.url.fl_str_mv |
https://repositorio.unal.edu.co/bitstream/unal/85427/1/license.txt https://repositorio.unal.edu.co/bitstream/unal/85427/2/1016058516.2024.pdf |
| bitstream.checksum.fl_str_mv |
eb34b1cf90b7e1103fc9dfd26be24b4a d71d42a8d65fe72af112be9053fd7acf |
| bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 |
| repository.name.fl_str_mv |
Repositorio Institucional Universidad Nacional de Colombia |
| repository.mail.fl_str_mv |
repositorio_nal@unal.edu.co |
| _version_ |
1806886303418023936 |
| spelling |
Reconocimiento 4.0 Internacionalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccesshttp://purl.org/coar/access_right/c_abf2Mojica Nava, Eduardo Alirio609c35fb4a7e288ee81a2ef0fb802397Yanguas Rojas, David Reinerio39949cab5cce1873511c7047f996ee6fPrograma de Investigacion sobre Adquisicion y Analisis de Señales Paas-Un0000-0001-5874-721XYanguas Rojas, DavidDavid R. Yanguas RojasDavid Yanguas-Rojas2024-01-24T20:37:35Z2024-01-24T20:37:35Z2023https://repositorio.unal.edu.co/handle/unal/85427Universidad Nacional de ColombiaRepositorio Institucional Universidad Nacional de Colombiahttps://repositorio.unal.edu.co/ilustraciones, diagramas, fotografíasEsta tesis presenta una estrategia de entrenamiento de robots que utiliza técnicas de aprendizaje artificial para optimizar el rendimiento de los robots en tareas complejas. Motivado por los impresionantes logros recientes en el aprendizaje automático, especialmente en juegos y escenarios virtuales, el proyecto tiene como objetivo explorar el potencial de estas técnicas para mejorar las capacidades de los robots más allá de la programación humana tradicional a pesar de las limitaciones impuestas por la brecha de la realidad. El caso de estudio seleccionado para esta investigación es la locomoción bípeda, ya que permite dilucidar los principales desafíos y ventajas de utilizar métodos de aprendizaje artificial para el aprendizaje de robots. La tesis identifica cuatro desafíos principales en este contexto: la variabilidad de los resultados obtenidos de los algoritmos de aprendizaje artificial, el alto costo y riesgo asociado con la realización de experimentos en robots reales, la brecha entre la simulación y el comportamiento del mundo real, y la necesidad de adaptar los patrones de movimiento humanos a los sistemas robóticos. La propuesta consiste en tres módulos principales para abordar estos desafíos: Enfoques de Control No Lineal, Aprendizaje por Imitación y Aprendizaje por Reforzamiento. El módulo de Enfoques de Control No Lineal establece una base al modelar robots y emplear técnicas de control bien establecidas. El módulo de Aprendizaje por Imitación utiliza la imitación para generar políticas iniciales basadas en datos de captura de movimiento de referencia o resultados preliminares de políticas para crear patrones de marcha similares a los humanos y factibles. El módulo de Aprendizaje por Refuerzos complementa el proceso mejorando de manera iterativa las políticas paramétricas, principalmente a través de la simulación pero con el rendimiento en el mundo real como objetivo final. Esta tesis enfatiza la modularidad del enfoque, permitiendo la implementación de los módulos individuales por separado o su combinación para determinar la estrategia más efectiva para diferentes escenarios de entrenamiento de robots. Al utilizar una combinación de técnicas de control establecidas, aprendizaje por imitación y aprendizaje por refuerzos, la estrategia de entrenamiento propuesta busca desbloquear el potencial para que los robots alcancen un rendimiento optimizado en tareas complejas, contribuyendo al avance de la inteligencia artificial en la robótica no solo en sistemas virtuales sino en sistemas reales.The thesis introduces a comprehensive robot training framework that utilizes artificial learning techniques to optimize robot performance in complex tasks. Motivated by recent impressive achievements in machine learning, particularly in games and virtual scenarios, the project aims to explore the potential of these techniques for improving robot capabilities beyond traditional human programming. The case study selected for this investigation is bipedal locomotion, as it allows for elucidating key challenges and advantages of using artificial learning methods for robot learning. The thesis identifies four primary challenges in this context: the variability of results obtained from artificial learning algorithms, the high cost and risk associated with conducting experiments on real robots, the reality gap between simulation and real-world behavior, and the need to adapt human motion patterns to robotic systems. The proposed approach consists of three main modules to address these challenges: Non-linear Control Approaches, Imitation Learning, and Reinforcement Learning. The Non-linear Control module establishes a foundation by modeling robots and employing well-established control techniques. The Imitation Learning module utilizes imitation to generate initial policies based on reference motion capture data or preliminary policy results to create feasible human-like gait patterns. The Reinforcement Learning module complements the process by iteratively improving parametric policies, primarily through simulation but ultimately with real-world performance as the ultimate goal. The thesis emphasizes the modularity of the approach, allowing for the implementation of individual modules separately or their combination to determine the most effective strategy for different robot training scenarios. By employing a combination of established control techniques, imitation learning, and reinforcement learning, the framework seeks to unlock the potential for robots to achieve optimized performances in complex tasks, contributing to the advancement of artificial intelligence in robotics.DoctoradoDoctor en ingeniería mecánica y mecatrónicaxxi, 158 páginasapplication/pdfengUniversidad Nacional de ColombiaBogotá - Ingeniería - Doctorado en Ingeniería - Ingeniería Mecánica y MecatrónicaFacultad de IngenieríaBogotá, ColombiaUniversidad Nacional de Colombia - Sede Bogotá620 - Ingeniería y operaciones afines::629 - Otras ramas de la ingenieríaAutomationRobóticaRoboticsAutomatizaciónAlgoritmosAlgorithmsReinforcement learningHumanoid RoboticsImitation LearningNon-Linear ControlRobot TrainingBipedal LocomotionHumanoid LocomotionArtificial Learning TechniquesReality GapSim to RealOvercoming the Reality Gap: Imitation and Reinforcement Learning Algorithms for Bipedal Robotic Locomotion ProblemsSuperando la brecha de la realidad: Algoritmos de aprendizaje por imitación y por refuerzos para problemas de locomoción robótica bípedaTrabajo de grado - Doctoradoinfo:eu-repo/semantics/doctoralThesisinfo:eu-repo/semantics/acceptedVersionhttp://purl.org/coar/resource_type/c_db06AudiovisualTexthttp://purl.org/redcol/resource_type/TD[Arcos-Legarda et al., 2019] Arcos-Legarda, J., Cortes-Romero, J., Beltran-Pulido, A., and Tovar, A. (2019). Hybrid disturbance rejection control of dynamic bipedal robots. Multibody System Dynamics, 46:281–306.[Belongie, 2023] Belongie, S. (2023). Rodrigues’ rotation formula. Created by Eric W. Weisstein.[Burda et al., 2018] Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., and Efros, A. A. (2018). Large-Scale Study of Curiosity-Driven Learning. ArXiv.[Carnegie Mellon University Graphics Lab, 2004] Carnegie Mellon University Graphics Lab (2004). CMU Motion Capture Database. The database was created with funding from NSF EIA-0196217.[Chebotar et al., 2018] Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., and Fox, D. (2018). Closing the sim-to-real loop: Adapting simulation randomization with real world experience.[Chernova and Thomaz, 2014] Chernova, S. and Thomaz, A. L. (2014). Robot Learning from Human Teachers. Synthesis Lectures on Artificial Intelligence and Machine Learning, 8(3):1–121.[Chernova and Veloso, 2010] Chernova, S. and Veloso, M. (2010). Confidence-based multirobot learning from demonstration. International Journal of Social Robotics, 2(2):195–215.[Chevallereau et al., 2014] Chevallereau, C., Sinnet, R. W., Ames, A. D., and Universit´e, L. (2014). Models feedback control and open problems of 3d bipedal robotic walking. Automatica.[Collins et al., 2005] Collins, S., Ruina, A., Tedrake, R., and Wisse, M. (2005). Efficient bipedal robots based on passive-dynamic walkers. Science, 307:1082–1085.[Ding et al., 2019] Ding, J., Zhou, C., and Xiao, X. (2019). Energy-Efficient Bipedal Gait Pattern Generation via CoM Acceleration Optimization. IEEE-RAS International Conference on Humanoid Robots, 2018-Novem:238–244.[Duan et al., 2017] Duan, Y., Andrychowicz, M., Stadie, B. C., Ho, J., Schneider, J., Sutskever, I., Abbeel, P., and Zaremba, W. (2017). One-Shot Imitation Learning. ArXiv.[Duan et al., 2016] Duan, Y., Chen, X., Houthooft, R., Schulman, J., and Abbeel, P. (2016). Benchmarking Deep Reinforcement Learning for Continuous Control. ArXiv.[Erez et al., 2015] Erez, T., Lowrey, K., Tassa, Y., Kumar, V., Kolev, S., and Todorov, E. (2015). An integrated system for real-time model predictive control of humanoid robots. Proceedings on IEEE-RAS International Conference on Humanoid Robots, 2015- Febru(February):292–299.[García et al., 1989] Garc´ıa, C. E., Prett, D. M., and Morari, M. (1989). Model predictive control: Theory and practice—a survey. Automatica, 25(3):335–348.[Google, 2015] Google (2015). AlphaGo — DeepMind. https://deepmind.com/research/ alphago/. accessed 20-May-2019.[Heess et al., 2017] Heess, N., TB, D., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Eslami, S. M. A., Riedmiller, M., and Silver, D. (2017). Emergence of locomotion behaviours in rich environments.[Ijspeert, 2008] Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: A review. Neural Networks, 21(4):642–653.[Jumper et al., 2021] Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., ˇ Z´ıdek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Petersen, S., Reiman, D., Clancy, E., Zielinski, M., Steinegger, M., Pacholska, M., Berghammer, T., Bodenstein, S., Silver, D., Vinyals, O., Senior, A. W., Kavukcuoglu, K., Kohli, P., and Hassabis, D. (2021). Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583 – 589. Cited by: 8028; All Open Access, Green Open Access, Hybrid Gold Open Access.[Kingma and Ba, 2015] Kingma, D. P. and Ba, J. L. (2015). Adam: A method for stochastic optimization.[Kobayashi et al., 2018] Kobayashi, T., Sekiyama, K., Hasegawa, Y., Aoyama, T., and Fukuda, T. (2018). Virtual-dynamics-based reference gait speed generator for limit-cycle-based bipedal gait. ROBOMECH Journal, 5(1).[Koenemann et al., 2014] Koenemann, J., Burget, F., and Bennewitz, M. (2014). Real-time imitation of human whole-body motions by humanoids. Proceedings - IEEE International Conference on Robotics and Automation, pages 2806–2812.[Koos et al., 2013] Koos, S., Mouret, J. B., and Doncieux, S. (2013). The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Transactions on Evolutionary Computation, 17(1):122–145.[Kuindersma et al., 2016] Kuindersma, S., Deits, R., Fallon, M., Valenzuela, A., Dai, H., Permenter, F., Koolen, T., Marion, P., and Tedrake, R. (2016). Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot. Autonomous Robots, 40:429–455.[Lee et al., 2019] Lee, K., Kim, S., Lim, S., Choi, S., and Oh, S. (2019). Tsallis Reinforcement Learning: A Unified Framework for Maximum Entropy Reinforcement Learning. ArXiv.[Lillicrap et al., 2015] Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. ArXiv.[Lillicrap et al., 2019] Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2019). Continuous control with deep reinforcement learning.[Liu et al., 2022] Liu, S., Lever, G., Wang, Z., Merel, J., Eslami, S., Hennes, D., Czarnecki, W., Tassa, Y., Omidshafiei, S., Abdolmaleki, A., Graepel, T., and Heess, N. (2022). From motor control to team play in simulated humanoid football. Science Robotics, 7.[Loudon et al., 2008] Loudon, J. K. J. K., Swift, M., and Bell, S. (2008). The clinical orthopedic assessment guide. Human Kinetics.[Luenberger and Ye, 2021] Luenberger, D. G. and Ye, Y. (2021). Penalty and barrier methods. International Series in Operations Research and Management Science, 228.[Ma et al., 2021] Ma, L.-K., Yang, Z., Xin, T., Guo, B., and Yin, K. (2021). Learning and exploring motor skills with spacetime bounds. Computer Graphics Forum, 40(2):251–263.[Ma et al., 2018] Ma, W.-L., Or, Y., and Ames, A. D. (2018). Dynamic Walking on Slippery Surfaces: Demonstrating Stable Bipedal Gaits with Planned Ground Slippage. Proceedings on 2019 International Conference on Robotics and Automation (ICRA).[Mania et al., 2018] Mania, H., Guy, A., and Recht, B. (2018). Simple random search provides a competitive approach to reinforcement learning. Advances in Neural Information Processing Systems 31 (NeurIPS 2018).[Mark W. Spong, 2020] Mark W. Spong, Seth Hutchinson, M. V. (2020). Robot modeling and control. JOHN WILEY and SONS, INC.[Merel et al., 2017] Merel, J., Tassa, Y., TB, D., Srinivasan, S., Lemmon, J., Wang, Z., Wayne, G., and Heess, N. (2017). Learning human behaviors from motion capture by adversarial imitation. eprint arXiv:1707.02201.[Mnih et al., 2016] Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., and Kavukcuoglu, K. (2016). Asynchronous Methods for Deep Reinforcement Learning.[Mnih et al., 2015] Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540):529 – 533. Cited by: 15639.[Müller et al., 2022] Müller, M., Jazdi, N., and Weyrich, M. (2022). Self-improving models for the intelligent digital twin: Towards closing the reality-to-simulation gap. IFACPapersOnLine, 55:126–131.Nehaniv and Dautenhahn, 2002] Nehaniv, C. L. and Dautenhahn, K. (2002). The Correspondence Problem, page 41–61. MIT Press, Cambridge, MA, USA.[Nguyen and La, 2019] Nguyen, H. and La, H. (2019). Review of Deep Reinforcement Learning for Robot Manipulation. Proceedings - 3rd IEEE International Conference on Robotic Computing, IRC 2019, pages 590–595.[Niu et al., 2022] Niu, J., Hu, Y., Li, W., Huang, G., Han, Y., and Li, X. (2022). Closing the dynamics gap via adversarial and reinforcement learning for high-speed racing. Proceedings of the International Joint Conference on Neural Networks, 2022-July.[OpenAI, 2018] OpenAI (2018). Openai five. https://blog.openai.com/openai-five/. accessed 20-May-2019.[Recht, 2018] Recht, B. (2018). A Tour of Reinforcement Learning: The View from Continuous Control. eprint arXiv:1806.09460.[Robbins and Monro, 1951] Robbins, H. and Monro, S. (1951). A Stochastic Approximation Method. The Annals of Mathematical Statistics, 22(3):400 – 407.[ROBOTIS, 2022a] ROBOTIS (2022a). Robotis engineer kit 1 e-manual (visited on 2022/04/01).[ROBOTIS, 2022b] ROBOTIS (2022b). Robotis Engineer Kit 2 E-Manual, (visited on 2022/04/01).[ROBOTIS, 2022c] ROBOTIS (2022c). Robotis mini e-manual (visited on 2022/04/01).[ROBOTIS, 2022d] ROBOTIS (2022d). Robotis xl-320 e-manual (visited on 2022/04/01).[Rodriguez et al., 2018] Rodriguez, D., Brandenburger, A., and Behnke, S. (2018). Combining Simulations and Real-robot Experiments for Bayesian Optimization of Bipedal Gait Stabilization. Lecture Notes in Computer Science book series.[Rosolia and Borrelli, 2016] Rosolia, U. and Borrelli, F. (2016). Learning Model Predictive Control for Iterative Tasks. eprint arXiv:1609.01387, 4.[Salvato et al., 2021] Salvato, E., Fenu, G., Medvet, E., and Pellegrino, F. A. (2021). Crossing the reality gap: A survey on sim-to-real transferability of robot controllers in reinforcement learning. IEEE Access, 9:153171–153187.[Schulman et al., 2015] Schulman, J., Levine, S., Moritz, P., Jordan, M. I., and Abbeel, P. (2015). Trust Region Policy Optimization. Proceedings of the 32nd International Conference on Machine Learning.[Siciliano and Khatib, 2016] Siciliano, B. and Khatib, O. (2016). Springer handbook of robotics. Springer eBooks.[Siegwart et al., 2011] Siegwart, R., Nourbakhsh, I. R., and Scaramuzza, D. (2011). Introduction to autonomous mobile robots. Intelligent robotics and autonomous agents. The MIT Press/Massachusetts Institute of Technology.[Simba et al., 2016] Simba, K. R., Uchiyama, N., and Sano, S. (2016). Real-time smooth trajectory generation for nonholonomic mobile robots using b´ezier curves. Robotics and Computer-Integrated Manufacturing, 41.[Singh et al., 2019] Singh, A., Yang, L., Hartikainen, K., Finn, C., and Levine, S. (2019). End-to-End Robotic Reinforcement Learning without Reward Engineering. Robotics: Science and Systems.[Sutton and Barto, 2008] Sutton, R. S. and Barto, A. G. (2008). Reinforcement Learning. The MIT Press, London, England, second edi edition.[´Swiechowski et al., 2022] ´Swiechowski, M., Godlewski, K., Sawicki, B., and Ma´ndziuk, J. (2022). Monte carlo tree search: a review of recent modifications and applications. Artificial Intelligence Review, 56(3):2497–2562.[Tan et al., 2018] Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., and Vanhoucke, V. (2018). Sim-to-Real: Learning Agile Locomotion For Quadruped Robots. eprint arXiv:1804.10332.[Tedrake, 2022] Tedrake, R. (2022). Underactuated Robotics. MIT.[Tevatia and Schaal, 2000] Tevatia, G. and Schaal, S. (2000). Inverse kinematics for humanoid robots. Proceedings-IEEE International Conference on Robotics and Automation, 1(April):294–299.[Thorp, 2023] Thorp, H. H. (2023). Chatgpt is fun, but not an author. Science, 379(6630):313; All Open Access, Bronze Open Access.[Tobin et al., 2017] Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. (2017). Domain randomization for transferring deep neural networks from simulation to the real world. IEEE International Conference on Intelligent Robots and Systems, 2017-Septe:23–30.[Todorov, 2019] Todorov, E. (2019). Mujoco: Modeling, simulation and visualization of multi-joint dynamics with contact. http://www.mujoco.org/book/index.html. accessed 20-May-2019.[Uchibe, 2018] Uchibe, E. (2018). Cooperative and competitive reinforcement and imitation learning for a mixture of heterogeneous learning modules. Frontiers in Neurorobotics, 12(SEP).[Vinyals et al., 2019] Vinyals, O., Babuschkin, I., Chung, J., Mathieu, M., Jaderberg, M., Czarnecki, W. M., Dudzik, A., Huang, A., Georgiev, P., Powell, R., Ewalds, T., Horgan, D., Kroiss, M., Danihelka, I., Agapiou, J., Oh, J., Dalibard, V., Choi, D., Sifre, L., Sulsky, Y., Vezhnevets, S., Molloy, J., Cai, T., Budden, D., Paine, T., Gulcehre, C., Wang, Z., Pfaff, T., Pohlen, T., Wu, Y., Yogatama, D., Cohen, J., McKinney, K., Smith, O., Schaul, T., Lillicrap, T., Apps, C., Kavukcuoglu, K., Hassabis, D., and Silver, D. (2019). AlphaStar: Mastering the Real-Time Strategy Game StarCraft II. https://deepmind.com/ blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/{}. accessed 20-May-2019.[Vukobratovic and Borovac, 2004] Vukobratovic, M. and Borovac, B. (2004). Zero-moment point — thirty five years of its life. International Journal of Humanoid Robotics, 01:157– 173.[Vukobratovic and Juricic, 1969] Vukobratovic, M. and Juricic, D. (1969). Contribution to the synthesis of biped gait. IEEE Transactions on Biomedical Engineering, BME-16(1):1– 6.[Wampler, 1986] Wampler, C. W. (1986). Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods. IEEE Transactions on Systems, Man and Cybernetics, 16.[Watkins and Dayan, 1992] Watkins, C. J. C. H. and Dayan, P. (1992). Q-learning. Machine Learning, 8(3):279–292.[Westervelt et al., 2003] Westervelt, E. R., Grizzle, J. W., and Koditschek, D. E. (2003). Hybrid zero dynamics of planar biped walkers. IEEE Transactions on Automatic Control, 48:42–56.[Xie et al., 2019] Xie, Z., Clary, P., Dao, J., Morais, P., Hurst, J., and van de Panne, M. (2019). Iterative reinforcement learning based design of dynamic locomotion skills for cassie. 3rd Conference on Robot Learning (CoRL 2019), Osaka, Japan.[Xie et al., 2020] Xie, Z., Ling, H. Y., Kim, N. H., and Panne, M. V. D. (2020). Allsteps: Curriculum-driven learning of stepping stone skills. ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA 2020, pages 213–224.[Zou et al., 2019] Zou, F., Shen, L., Jie, Z., Zhang, W., and Liu, W. (2019). A sufficient condition for convergences of adam and rmsprop. volume 2019-June.EstudiantesInvestigadoresLICENSElicense.txtlicense.txttext/plain; charset=utf-85879https://repositorio.unal.edu.co/bitstream/unal/85427/1/license.txteb34b1cf90b7e1103fc9dfd26be24b4aMD51ORIGINAL1016058516.2024.pdf1016058516.2024.pdfTesis de Doctorado en Ingeniería Mecánica y Mecatrónicaapplication/pdf13748562https://repositorio.unal.edu.co/bitstream/unal/85427/2/1016058516.2024.pdfd71d42a8d65fe72af112be9053fd7acfMD52unal/85427oai:repositorio.unal.edu.co:unal/854272024-01-24 15:37:37.168Repositorio Institucional Universidad Nacional de Colombiarepositorio_nal@unal.edu.coUEFSVEUgMS4gVMOJUk1JTk9TIERFIExBIExJQ0VOQ0lBIFBBUkEgUFVCTElDQUNJw5NOIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KCkxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgbG9zIGRlcmVjaG9zIHBhdHJpbW9uaWFsZXMgZGUgYXV0b3IsIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEsIGxpbWl0YWRhIHkgZ3JhdHVpdGEgc29icmUgbGEgb2JyYSBxdWUgc2UgaW50ZWdyYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsLCBiYWpvIGxvcyBzaWd1aWVudGVzIHTDqXJtaW5vczoKCgphKQlMb3MgYXV0b3JlcyB5L28gbG9zIHRpdHVsYXJlcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBzb2JyZSBsYSBvYnJhIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEgcGFyYSByZWFsaXphciBsb3Mgc2lndWllbnRlcyBhY3RvcyBzb2JyZSBsYSBvYnJhOiBpKSByZXByb2R1Y2lyIGxhIG9icmEgZGUgbWFuZXJhIGRpZ2l0YWwsIHBlcm1hbmVudGUgbyB0ZW1wb3JhbCwgaW5jbHV5ZW5kbyBlbCBhbG1hY2VuYW1pZW50byBlbGVjdHLDs25pY28sIGFzw60gY29tbyBjb252ZXJ0aXIgZWwgZG9jdW1lbnRvIGVuIGVsIGN1YWwgc2UgZW5jdWVudHJhIGNvbnRlbmlkYSBsYSBvYnJhIGEgY3VhbHF1aWVyIG1lZGlvIG8gZm9ybWF0byBleGlzdGVudGUgYSBsYSBmZWNoYSBkZSBsYSBzdXNjcmlwY2nDs24gZGUgbGEgcHJlc2VudGUgbGljZW5jaWEsIHkgaWkpIGNvbXVuaWNhciBhbCBww7pibGljbyBsYSBvYnJhIHBvciBjdWFscXVpZXIgbWVkaW8gbyBwcm9jZWRpbWllbnRvLCBlbiBtZWRpb3MgYWzDoW1icmljb3MgbyBpbmFsw6FtYnJpY29zLCBpbmNsdXllbmRvIGxhIHB1ZXN0YSBhIGRpc3Bvc2ljacOzbiBlbiBhY2Nlc28gYWJpZXJ0by4gQWRpY2lvbmFsIGEgbG8gYW50ZXJpb3IsIGVsIGF1dG9yIHkvbyB0aXR1bGFyIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgcGFyYSBxdWUsIGVuIGxhIHJlcHJvZHVjY2nDs24geSBjb211bmljYWNpw7NuIGFsIHDDumJsaWNvIHF1ZSBsYSBVbml2ZXJzaWRhZCByZWFsaWNlIHNvYnJlIGxhIG9icmEsIGhhZ2EgbWVuY2nDs24gZGUgbWFuZXJhIGV4cHJlc2EgYWwgdGlwbyBkZSBsaWNlbmNpYSBDcmVhdGl2ZSBDb21tb25zIGJham8gbGEgY3VhbCBlbCBhdXRvciB5L28gdGl0dWxhciBkZXNlYSBvZnJlY2VyIHN1IG9icmEgYSBsb3MgdGVyY2Vyb3MgcXVlIGFjY2VkYW4gYSBkaWNoYSBvYnJhIGEgdHJhdsOpcyBkZWwgUmVwb3NpdG9yaW8gSW5zdGl0dWNpb25hbCwgY3VhbmRvIHNlYSBlbCBjYXNvLiBFbCBhdXRvciB5L28gdGl0dWxhciBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBwb2Ryw6EgZGFyIHBvciB0ZXJtaW5hZGEgbGEgcHJlc2VudGUgbGljZW5jaWEgbWVkaWFudGUgc29saWNpdHVkIGVsZXZhZGEgYSBsYSBEaXJlY2Npw7NuIE5hY2lvbmFsIGRlIEJpYmxpb3RlY2FzIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLiAKCmIpIAlMb3MgYXV0b3JlcyB5L28gdGl0dWxhcmVzIGRlIGxvcyBkZXJlY2hvcyBwYXRyaW1vbmlhbGVzIGRlIGF1dG9yIHNvYnJlIGxhIG9icmEgY29uZmllcmVuIGxhIGxpY2VuY2lhIHNlw7FhbGFkYSBlbiBlbCBsaXRlcmFsIGEpIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8gcG9yIGVsIHRpZW1wbyBkZSBwcm90ZWNjacOzbiBkZSBsYSBvYnJhIGVuIHRvZG9zIGxvcyBwYcOtc2VzIGRlbCBtdW5kbywgZXN0byBlcywgc2luIGxpbWl0YWNpw7NuIHRlcnJpdG9yaWFsIGFsZ3VuYS4KCmMpCUxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBtYW5pZmllc3RhbiBlc3RhciBkZSBhY3VlcmRvIGNvbiBxdWUgbGEgcHJlc2VudGUgbGljZW5jaWEgc2Ugb3RvcmdhIGEgdMOtdHVsbyBncmF0dWl0bywgcG9yIGxvIHRhbnRvLCByZW51bmNpYW4gYSByZWNpYmlyIGN1YWxxdWllciByZXRyaWJ1Y2nDs24gZWNvbsOzbWljYSBvIGVtb2x1bWVudG8gYWxndW5vIHBvciBsYSBwdWJsaWNhY2nDs24sIGRpc3RyaWJ1Y2nDs24sIGNvbXVuaWNhY2nDs24gcMO6YmxpY2EgeSBjdWFscXVpZXIgb3RybyB1c28gcXVlIHNlIGhhZ2EgZW4gbG9zIHTDqXJtaW5vcyBkZSBsYSBwcmVzZW50ZSBsaWNlbmNpYSB5IGRlIGxhIGxpY2VuY2lhIENyZWF0aXZlIENvbW1vbnMgY29uIHF1ZSBzZSBwdWJsaWNhLgoKZCkJUXVpZW5lcyBmaXJtYW4gZWwgcHJlc2VudGUgZG9jdW1lbnRvIGRlY2xhcmFuIHF1ZSBwYXJhIGxhIGNyZWFjacOzbiBkZSBsYSBvYnJhLCBubyBzZSBoYW4gdnVsbmVyYWRvIGxvcyBkZXJlY2hvcyBkZSBwcm9waWVkYWQgaW50ZWxlY3R1YWwsIGluZHVzdHJpYWwsIG1vcmFsZXMgeSBwYXRyaW1vbmlhbGVzIGRlIHRlcmNlcm9zLiBEZSBvdHJhIHBhcnRlLCAgcmVjb25vY2VuIHF1ZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhY3TDumEgY29tbyB1biB0ZXJjZXJvIGRlIGJ1ZW5hIGZlIHkgc2UgZW5jdWVudHJhIGV4ZW50YSBkZSBjdWxwYSBlbiBjYXNvIGRlIHByZXNlbnRhcnNlIGFsZ8O6biB0aXBvIGRlIHJlY2xhbWFjacOzbiBlbiBtYXRlcmlhIGRlIGRlcmVjaG9zIGRlIGF1dG9yIG8gcHJvcGllZGFkIGludGVsZWN0dWFsIGVuIGdlbmVyYWwuIFBvciBsbyB0YW50bywgbG9zIGZpcm1hbnRlcyAgYWNlcHRhbiBxdWUgY29tbyB0aXR1bGFyZXMgw7puaWNvcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciwgYXN1bWlyw6FuIHRvZGEgbGEgcmVzcG9uc2FiaWxpZGFkIGNpdmlsLCBhZG1pbmlzdHJhdGl2YSB5L28gcGVuYWwgcXVlIHB1ZWRhIGRlcml2YXJzZSBkZSBsYSBwdWJsaWNhY2nDs24gZGUgbGEgb2JyYS4gIAoKZikJQXV0b3JpemFuIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgaW5jbHVpciBsYSBvYnJhIGVuIGxvcyBhZ3JlZ2Fkb3JlcyBkZSBjb250ZW5pZG9zLCBidXNjYWRvcmVzIGFjYWTDqW1pY29zLCBtZXRhYnVzY2Fkb3Jlcywgw61uZGljZXMgeSBkZW3DoXMgbWVkaW9zIHF1ZSBzZSBlc3RpbWVuIG5lY2VzYXJpb3MgcGFyYSBwcm9tb3ZlciBlbCBhY2Nlc28geSBjb25zdWx0YSBkZSBsYSBtaXNtYS4gCgpnKQlFbiBlbCBjYXNvIGRlIGxhcyB0ZXNpcyBjcmVhZGFzIHBhcmEgb3B0YXIgZG9ibGUgdGl0dWxhY2nDs24sIGxvcyBmaXJtYW50ZXMgc2Vyw6FuIGxvcyByZXNwb25zYWJsZXMgZGUgY29tdW5pY2FyIGEgbGFzIGluc3RpdHVjaW9uZXMgbmFjaW9uYWxlcyBvIGV4dHJhbmplcmFzIGVuIGNvbnZlbmlvLCBsYXMgbGljZW5jaWFzIGRlIGFjY2VzbyBhYmllcnRvIENyZWF0aXZlIENvbW1vbnMgeSBhdXRvcml6YWNpb25lcyBhc2lnbmFkYXMgYSBzdSBvYnJhIHBhcmEgbGEgcHVibGljYWNpw7NuIGVuIGVsIFJlcG9zaXRvcmlvIEluc3RpdHVjaW9uYWwgVU5BTCBkZSBhY3VlcmRvIGNvbiBsYXMgZGlyZWN0cmljZXMgZGUgbGEgUG9sw610aWNhIEdlbmVyYWwgZGUgbGEgQmlibGlvdGVjYSBEaWdpdGFsLgoKCmgpCVNlIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgY29tbyByZXNwb25zYWJsZSBkZWwgdHJhdGFtaWVudG8gZGUgZGF0b3MgcGVyc29uYWxlcywgZGUgYWN1ZXJkbyBjb24gbGEgbGV5IDE1ODEgZGUgMjAxMiBlbnRlbmRpZW5kbyBxdWUgc2UgZW5jdWVudHJhbiBiYWpvIG1lZGlkYXMgcXVlIGdhcmFudGl6YW4gbGEgc2VndXJpZGFkLCBjb25maWRlbmNpYWxpZGFkIGUgaW50ZWdyaWRhZCwgeSBzdSB0cmF0YW1pZW50byB0aWVuZSB1bmEgZmluYWxpZGFkIGhpc3TDs3JpY2EsIGVzdGFkw61zdGljYSBvIGNpZW50w61maWNhIHNlZ8O6biBsbyBkaXNwdWVzdG8gZW4gbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMuCgoKClBBUlRFIDIuIEFVVE9SSVpBQ0nDk04gUEFSQSBQVUJMSUNBUiBZIFBFUk1JVElSIExBIENPTlNVTFRBIFkgVVNPIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KClNlIGF1dG9yaXphIGxhIHB1YmxpY2FjacOzbiBlbGVjdHLDs25pY2EsIGNvbnN1bHRhIHkgdXNvIGRlIGxhIG9icmEgcG9yIHBhcnRlIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgZGUgc3VzIHVzdWFyaW9zIGRlIGxhIHNpZ3VpZW50ZSBtYW5lcmE6CgphLglDb25jZWRvIGxpY2VuY2lhIGVuIGxvcyB0w6lybWlub3Mgc2XDsWFsYWRvcyBlbiBsYSBwYXJ0ZSAxIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8sIGNvbiBlbCBvYmpldGl2byBkZSBxdWUgbGEgb2JyYSBlbnRyZWdhZGEgc2VhIHB1YmxpY2FkYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgcHVlc3RhIGEgZGlzcG9zaWNpw7NuIGVuIGFjY2VzbyBhYmllcnRvIHBhcmEgc3UgY29uc3VsdGEgcG9yIGxvcyB1c3VhcmlvcyBkZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSAgYSB0cmF2w6lzIGRlIGludGVybmV0LgoKCgpQQVJURSAzIEFVVE9SSVpBQ0nDk04gREUgVFJBVEFNSUVOVE8gREUgREFUT1MgUEVSU09OQUxFUy4KCkxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLCBjb21vIHJlc3BvbnNhYmxlIGRlbCBUcmF0YW1pZW50byBkZSBEYXRvcyBQZXJzb25hbGVzLCBpbmZvcm1hIHF1ZSBsb3MgZGF0b3MgZGUgY2Fyw6FjdGVyIHBlcnNvbmFsIHJlY29sZWN0YWRvcyBtZWRpYW50ZSBlc3RlIGZvcm11bGFyaW8sIHNlIGVuY3VlbnRyYW4gYmFqbyBtZWRpZGFzIHF1ZSBnYXJhbnRpemFuIGxhIHNlZ3VyaWRhZCwgY29uZmlkZW5jaWFsaWRhZCBlIGludGVncmlkYWQgeSBzdSB0cmF0YW1pZW50byBzZSByZWFsaXphIGRlIGFjdWVyZG8gYWwgY3VtcGxpbWllbnRvIG5vcm1hdGl2byBkZSBsYSBMZXkgMTU4MSBkZSAyMDEyIHkgZGUgbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMgZGUgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEuIFB1ZWRlIGVqZXJjZXIgc3VzIGRlcmVjaG9zIGNvbW8gdGl0dWxhciBhIGNvbm9jZXIsIGFjdHVhbGl6YXIsIHJlY3RpZmljYXIgeSByZXZvY2FyIGxhcyBhdXRvcml6YWNpb25lcyBkYWRhcyBhIGxhcyBmaW5hbGlkYWRlcyBhcGxpY2FibGVzIGEgdHJhdsOpcyBkZSBsb3MgY2FuYWxlcyBkaXNwdWVzdG9zIHkgZGlzcG9uaWJsZXMgZW4gd3d3LnVuYWwuZWR1LmNvIG8gZS1tYWlsOiBwcm90ZWNkYXRvc19uYUB1bmFsLmVkdS5jbyIKClRlbmllbmRvIGVuIGN1ZW50YSBsbyBhbnRlcmlvciwgYXV0b3Jpem8gZGUgbWFuZXJhIHZvbHVudGFyaWEsIHByZXZpYSwgZXhwbMOtY2l0YSwgaW5mb3JtYWRhIGUgaW5lcXXDrXZvY2EgYSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhIHRyYXRhciBsb3MgZGF0b3MgcGVyc29uYWxlcyBkZSBhY3VlcmRvIGNvbiBsYXMgZmluYWxpZGFkZXMgZXNwZWPDrWZpY2FzIHBhcmEgZWwgZGVzYXJyb2xsbyB5IGVqZXJjaWNpbyBkZSBsYXMgZnVuY2lvbmVzIG1pc2lvbmFsZXMgZGUgZG9jZW5jaWEsIGludmVzdGlnYWNpw7NuIHkgZXh0ZW5zacOzbiwgYXPDrSBjb21vIGxhcyByZWxhY2lvbmVzIGFjYWTDqW1pY2FzLCBsYWJvcmFsZXMsIGNvbnRyYWN0dWFsZXMgeSB0b2RhcyBsYXMgZGVtw6FzIHJlbGFjaW9uYWRhcyBjb24gZWwgb2JqZXRvIHNvY2lhbCBkZSBsYSBVbml2ZXJzaWRhZC4gCgo= |