AI-Generated code detection: an examination of current tools in education
This document explores the challenge of detecting AI-generated Python code in education, highlighting limitations of current detection tools, especially against simple obfuscation techniques. It emphasizes the need for advanced, resilient detection methods and ethical AI use in academic settings.
- Autores:
-
Cuellar Argotty, Juan Esteban
- Tipo de recurso:
- Trabajo de grado de pregrado
- Fecha de publicación:
- 2025
- Institución:
- Universidad de los Andes
- Repositorio:
- Séneca: repositorio Uniandes
- Idioma:
- eng
- OAI Identifier:
- oai:repositorio.uniandes.edu.co:1992/75503
- Acceso en línea:
- https://hdl.handle.net/1992/75503
- Palabra clave:
- AI-generated code
AI-Generated Code Detection
Software Engineering Education
Ingeniería
- Rights
- embargoedAccess
- License
- Attribution-NonCommercial-NoDerivatives 4.0 International
| id |
UNIANDES2_01d72919cbdb8612ea3408b7a5af45b4 |
|---|---|
| oai_identifier_str |
oai:repositorio.uniandes.edu.co:1992/75503 |
| network_acronym_str |
UNIANDES2 |
| network_name_str |
Séneca: repositorio Uniandes |
| repository_id_str |
|
| dc.title.none.fl_str_mv |
AI-Generated code detection: an examination of current tools in education |
| title |
AI-Generated code detection: an examination of current tools in education |
| spellingShingle |
AI-Generated code detection: an examination of current tools in education AI-generated code AI-Generated Code Detection Software Engineering Education Ingeniería |
| title_short |
AI-Generated code detection: an examination of current tools in education |
| title_full |
AI-Generated code detection: an examination of current tools in education |
| title_fullStr |
AI-Generated code detection: an examination of current tools in education |
| title_full_unstemmed |
AI-Generated code detection: an examination of current tools in education |
| title_sort |
AI-Generated code detection: an examination of current tools in education |
| dc.creator.fl_str_mv |
Cuellar Argotty, Juan Esteban |
| dc.contributor.advisor.none.fl_str_mv |
Manrique Piramanrique, Rubén Francisco |
| dc.contributor.author.none.fl_str_mv |
Cuellar Argotty, Juan Esteban |
| dc.contributor.researchgroup.none.fl_str_mv |
Facultad de Ingeniería |
| dc.subject.keyword.none.fl_str_mv |
AI-generated code AI-Generated Code Detection Software Engineering Education |
| topic |
AI-generated code AI-Generated Code Detection Software Engineering Education Ingeniería |
| dc.subject.themes.none.fl_str_mv |
Ingeniería |
| description |
This document explores the challenge of detecting AI-generated Python code in education, highlighting limitations of current detection tools, especially against simple obfuscation techniques. It emphasizes the need for advanced, resilient detection methods and ethical AI use in academic settings. |
| publishDate |
2025 |
| dc.date.accessioned.none.fl_str_mv |
2025-01-20T20:03:03Z |
| dc.date.issued.none.fl_str_mv |
2025-01-20 |
| dc.date.accepted.none.fl_str_mv |
2025-01-20 |
| dc.date.available.none.fl_str_mv |
2026-01-19 |
| dc.type.none.fl_str_mv |
Trabajo de grado - Pregrado |
| dc.type.driver.none.fl_str_mv |
info:eu-repo/semantics/bachelorThesis |
| dc.type.version.none.fl_str_mv |
info:eu-repo/semantics/acceptedVersion |
| dc.type.coar.none.fl_str_mv |
http://purl.org/coar/resource_type/c_7a1f |
| dc.type.content.none.fl_str_mv |
Text |
| dc.type.redcol.none.fl_str_mv |
http://purl.org/redcol/resource_type/TP |
| format |
http://purl.org/coar/resource_type/c_7a1f |
| status_str |
acceptedVersion |
| dc.identifier.uri.none.fl_str_mv |
https://hdl.handle.net/1992/75503 |
| dc.identifier.instname.none.fl_str_mv |
instname:Universidad de los Andes |
| dc.identifier.reponame.none.fl_str_mv |
reponame:Repositorio Institucional Séneca |
| dc.identifier.repourl.none.fl_str_mv |
repourl:https://repositorio.uniandes.edu.co/ |
| url |
https://hdl.handle.net/1992/75503 |
| identifier_str_mv |
instname:Universidad de los Andes reponame:Repositorio Institucional Séneca repourl:https://repositorio.uniandes.edu.co/ |
| dc.language.iso.none.fl_str_mv |
eng |
| language |
eng |
| dc.relation.references.none.fl_str_mv |
Uchendu, Adaku, Venkatraman, Saranya, Le, Thai, and Lee, Dongwon. "Catch Me If You GPT: Tutorial on Deepfake Texts". Available at: https://aclanthology.org/2024.naacl-tutorials.1.pdf Weber-Wulff, Debora, Anohina-Naumeca, Alla, and Bjelobaba, Sonja. "Testing of detection tools for AI-generated text". SpringerLink, 2023. Available at: https://link.springer.com/article/10.1007/s40979-023-00146-z Mitchell, Eric, Lee, Yoonho, Khazatsky, Alexander, Manning, Christopher D., and Finn, Chelsea. "DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature". Available at: https://openreview.net/pdf?id=UiAyIILXRd Pan, Wei Hung, Chok, Ming Jie, and Wong, Jonathan Leong Shan. "Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education". IEEE Xplore, 2023. Available at: https://ieeexplore.ieee.org/document/10554754 Li, Yafu, Li, Qintong, and Cui, Leyang. "MAGE: Machine-generated Text Detection in the Wild". Available at: https://arxiv.org/pdf/2305.13242 Xu, Zhenyu, and Sheng, Victor S. "Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models". AAAI Conference on Artificial Intelligence, 2023. Available at: https://ojs.aaai.org/index.php/AAAI/article/view/30361 New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution. Available at: https://www.mdpi.com/2071-1050/15/16/12451 The role and impact of ChatGPT in educational practices: insights from an Australian higher education case study. Available at: https://link.springer.com/article/10.1007/s44217-024-00126-6 |
| dc.rights.en.fl_str_mv |
Attribution-NonCommercial-NoDerivatives 4.0 International |
| dc.rights.uri.none.fl_str_mv |
http://creativecommons.org/licenses/by-nc-nd/4.0/ |
| dc.rights.accessrights.none.fl_str_mv |
info:eu-repo/semantics/embargoedAccess |
| dc.rights.coar.none.fl_str_mv |
http://purl.org/coar/access_right/c_f1cf |
| rights_invalid_str_mv |
Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/ http://purl.org/coar/access_right/c_f1cf |
| eu_rights_str_mv |
embargoedAccess |
| dc.format.extent.none.fl_str_mv |
29 páginas |
| dc.format.mimetype.none.fl_str_mv |
application/pdf |
| dc.publisher.none.fl_str_mv |
Universidad de los Andes |
| dc.publisher.program.none.fl_str_mv |
Ingeniería de Sistemas y Computación |
| dc.publisher.faculty.none.fl_str_mv |
Facultad de Ingeniería |
| dc.publisher.department.none.fl_str_mv |
Departamento de Ingeniería de Sistemas y Computación |
| publisher.none.fl_str_mv |
Universidad de los Andes |
| institution |
Universidad de los Andes |
| bitstream.url.fl_str_mv |
https://repositorio.uniandes.edu.co/bitstreams/4e5af9cf-bebe-437d-a424-46f0adfe24c0/download https://repositorio.uniandes.edu.co/bitstreams/da71716d-0ca4-4da3-b714-b603b8e9f5fe/download https://repositorio.uniandes.edu.co/bitstreams/c503e966-a54b-4997-88c0-1b47b1ea7d79/download https://repositorio.uniandes.edu.co/bitstreams/feba8d77-7eff-4cd1-8580-8487723459f6/download https://repositorio.uniandes.edu.co/bitstreams/892d5b0d-4b8e-4310-bb0a-4c0f2230416d/download https://repositorio.uniandes.edu.co/bitstreams/5629c54e-2ac0-49b4-8fc7-8a3366026683/download https://repositorio.uniandes.edu.co/bitstreams/91b9db14-977d-4d53-aea7-6596cdee8ba5/download https://repositorio.uniandes.edu.co/bitstreams/eeccc7c6-83bb-452d-80e9-c8dea37499cf/download |
| bitstream.checksum.fl_str_mv |
566f8ddec659cf94f932ffc49c8433cc 5b266caca553f4d58de8faa550244062 4460e5956bc1d1639be9ae6146a50347 ae9e573a68e7f92501b6913cc846c39f 369d1c3b0003e205e42f386962183dfc 530d320eb9a3b695b3582e4fc981c68e ac096d59857f8693638ebd3da10f62ca 40b10268eab9b784d49ed4542a4f07c3 |
| bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 MD5 MD5 MD5 MD5 MD5 MD5 |
| repository.name.fl_str_mv |
Repositorio institucional Séneca |
| repository.mail.fl_str_mv |
adminrepositorio@uniandes.edu.co |
| _version_ |
1828159293537386496 |
| spelling |
Manrique Piramanrique, Rubén Franciscovirtual::22239-1Cuellar Argotty, Juan EstebanFacultad de Ingeniería2025-01-20T20:03:03Z2026-01-192025-01-202025-01-20https://hdl.handle.net/1992/75503instname:Universidad de los Andesreponame:Repositorio Institucional Sénecarepourl:https://repositorio.uniandes.edu.co/This document explores the challenge of detecting AI-generated Python code in education, highlighting limitations of current detection tools, especially against simple obfuscation techniques. It emphasizes the need for advanced, resilient detection methods and ethical AI use in academic settings.This document explores the challenge of detecting AI-generated Python code within educational settings, focusing on first-semester student solutions on the Senecode platform. It outlines the creation of a dataset combining both human-written and AI-generated code (across multiple obfuscation variants) and evaluates seven widely used AI detectors. Despite each tool’s strengths in certain areas—such as high precision or high recall—none consistently excels, and simple code modifications substantially reduce detection accuracy. The study underscores the trade-off between minimizing false positives and maximizing true detection, highlighting the risk of unjustly penalizing students or overlooking AI misuse. Recommendations include developing more advanced, code-specific detection methods, employing a multi-layer approach that integrates human oversight, and fostering ethical AI use through clear academic policies.Pregrado29 páginasapplication/pdfengUniversidad de los AndesIngeniería de Sistemas y ComputaciónFacultad de IngenieríaDepartamento de Ingeniería de Sistemas y ComputaciónAttribution-NonCommercial-NoDerivatives 4.0 Internationalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/embargoedAccesshttp://purl.org/coar/access_right/c_f1cfAI-Generated code detection: an examination of current tools in educationTrabajo de grado - Pregradoinfo:eu-repo/semantics/bachelorThesisinfo:eu-repo/semantics/acceptedVersionhttp://purl.org/coar/resource_type/c_7a1fTexthttp://purl.org/redcol/resource_type/TPAI-generated codeAI-Generated Code DetectionSoftware Engineering EducationIngenieríaUchendu, Adaku, Venkatraman, Saranya, Le, Thai, and Lee, Dongwon. "Catch Me If You GPT: Tutorial on Deepfake Texts". Available at: https://aclanthology.org/2024.naacl-tutorials.1.pdfWeber-Wulff, Debora, Anohina-Naumeca, Alla, and Bjelobaba, Sonja. "Testing of detection tools for AI-generated text". SpringerLink, 2023. Available at: https://link.springer.com/article/10.1007/s40979-023-00146-zMitchell, Eric, Lee, Yoonho, Khazatsky, Alexander, Manning, Christopher D., and Finn, Chelsea. "DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature". Available at: https://openreview.net/pdf?id=UiAyIILXRdPan, Wei Hung, Chok, Ming Jie, and Wong, Jonathan Leong Shan. "Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education". IEEE Xplore, 2023. Available at: https://ieeexplore.ieee.org/document/10554754Li, Yafu, Li, Qintong, and Cui, Leyang. "MAGE: Machine-generated Text Detection in the Wild". Available at: https://arxiv.org/pdf/2305.13242Xu, Zhenyu, and Sheng, Victor S. "Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models". AAAI Conference on Artificial Intelligence, 2023. Available at: https://ojs.aaai.org/index.php/AAAI/article/view/30361New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution. Available at: https://www.mdpi.com/2071-1050/15/16/12451The role and impact of ChatGPT in educational practices: insights from an Australian higher education case study. Available at: https://link.springer.com/article/10.1007/s44217-024-00126-6202014258Publication9f6e12e0-098e-4548-ab81-75552e8385e7virtual::22239-19f6e12e0-098e-4548-ab81-75552e8385e7virtual::22239-1ORIGINALAI-Generated Code Detection: An Examination of Current Tools in Education.pdfAI-Generated Code Detection: An Examination of Current Tools in Education.pdfRestricción de acceso hasta el año 2026application/pdf600011https://repositorio.uniandes.edu.co/bitstreams/4e5af9cf-bebe-437d-a424-46f0adfe24c0/download566f8ddec659cf94f932ffc49c8433ccMD521ef29e67-d027-45df-b771-f560a6ba7c22_signed.pdf1ef29e67-d027-45df-b771-f560a6ba7c22_signed.pdfHIDEapplication/pdf205328https://repositorio.uniandes.edu.co/bitstreams/da71716d-0ca4-4da3-b714-b603b8e9f5fe/download5b266caca553f4d58de8faa550244062MD51CC-LICENSElicense_rdflicense_rdfapplication/rdf+xml; charset=utf-8805https://repositorio.uniandes.edu.co/bitstreams/c503e966-a54b-4997-88c0-1b47b1ea7d79/download4460e5956bc1d1639be9ae6146a50347MD53LICENSElicense.txtlicense.txttext/plain; charset=utf-82535https://repositorio.uniandes.edu.co/bitstreams/feba8d77-7eff-4cd1-8580-8487723459f6/downloadae9e573a68e7f92501b6913cc846c39fMD54TEXTAI-Generated Code Detection: An Examination of Current Tools in Education.pdf.txtAI-Generated Code Detection: An Examination of Current Tools in Education.pdf.txtExtracted texttext/plain72436https://repositorio.uniandes.edu.co/bitstreams/892d5b0d-4b8e-4310-bb0a-4c0f2230416d/download369d1c3b0003e205e42f386962183dfcMD551ef29e67-d027-45df-b771-f560a6ba7c22_signed.pdf.txt1ef29e67-d027-45df-b771-f560a6ba7c22_signed.pdf.txtExtracted texttext/plain1176https://repositorio.uniandes.edu.co/bitstreams/5629c54e-2ac0-49b4-8fc7-8a3366026683/download530d320eb9a3b695b3582e4fc981c68eMD57THUMBNAILAI-Generated Code Detection: An Examination of Current Tools in Education.pdf.jpgAI-Generated Code Detection: An Examination of Current Tools in Education.pdf.jpgGenerated Thumbnailimage/jpeg5894https://repositorio.uniandes.edu.co/bitstreams/91b9db14-977d-4d53-aea7-6596cdee8ba5/downloadac096d59857f8693638ebd3da10f62caMD561ef29e67-d027-45df-b771-f560a6ba7c22_signed.pdf.jpg1ef29e67-d027-45df-b771-f560a6ba7c22_signed.pdf.jpgGenerated Thumbnailimage/jpeg11017https://repositorio.uniandes.edu.co/bitstreams/eeccc7c6-83bb-452d-80e9-c8dea37499cf/download40b10268eab9b784d49ed4542a4f07c3MD581992/75503oai:repositorio.uniandes.edu.co:1992/755032025-03-05 10:02:02.891http://creativecommons.org/licenses/by-nc-nd/4.0/Attribution-NonCommercial-NoDerivatives 4.0 Internationalrestrictedhttps://repositorio.uniandes.edu.coRepositorio institucional Sénecaadminrepositorio@uniandes.edu.coPGgzPjxzdHJvbmc+RGVzY2FyZ28gZGUgUmVzcG9uc2FiaWxpZGFkIC0gTGljZW5jaWEgZGUgQXV0b3JpemFjacOzbjwvc3Ryb25nPjwvaDM+CjxwPjxzdHJvbmc+UG9yIGZhdm9yIGxlZXIgYXRlbnRhbWVudGUgZXN0ZSBkb2N1bWVudG8gcXVlIHBlcm1pdGUgYWwgUmVwb3NpdG9yaW8gSW5zdGl0dWNpb25hbCBTw6luZWNhIHJlcHJvZHVjaXIgeSBkaXN0cmlidWlyIGxvcyByZWN1cnNvcyBkZSBpbmZvcm1hY2nDs24gZGVwb3NpdGFkb3MgbWVkaWFudGUgbGEgYXV0b3JpemFjacOzbiBkZSBsb3Mgc2lndWllbnRlcyB0w6lybWlub3M6PC9zdHJvbmc+PC9wPgo8cD5Db25jZWRhIGxhIGxpY2VuY2lhIGRlIGRlcMOzc2l0byBlc3TDoW5kYXIgc2VsZWNjaW9uYW5kbyBsYSBvcGNpw7NuIDxzdHJvbmc+J0FjZXB0YXIgbG9zIHTDqXJtaW5vcyBhbnRlcmlvcm1lbnRlIGRlc2NyaXRvcyc8L3N0cm9uZz4geSBjb250aW51YXIgZWwgcHJvY2VzbyBkZSBlbnbDrW8gbWVkaWFudGUgZWwgYm90w7NuIDxzdHJvbmc+J1NpZ3VpZW50ZScuPC9zdHJvbmc+PC9wPgo8aHI+CjxwPllvLCBlbiBtaSBjYWxpZGFkIGRlIGF1dG9yIGRlbCB0cmFiYWpvIGRlIHRlc2lzLCBtb25vZ3JhZsOtYSBvIHRyYWJham8gZGUgZ3JhZG8sIGhhZ28gZW50cmVnYSBkZWwgZWplbXBsYXIgcmVzcGVjdGl2byB5IGRlIHN1cyBhbmV4b3MgZGUgc2VyIGVsIGNhc28sIGVuIGZvcm1hdG8gZGlnaXRhbCB5L28gZWxlY3Ryw7NuaWNvIHkgYXV0b3Jpem8gYSBsYSBVbml2ZXJzaWRhZCBkZSBsb3MgQW5kZXMgcGFyYSBxdWUgcmVhbGljZSBsYSBwdWJsaWNhY2nDs24gZW4gZWwgU2lzdGVtYSBkZSBCaWJsaW90ZWNhcyBvIGVuIGN1YWxxdWllciBvdHJvIHNpc3RlbWEgbyBiYXNlIGRlIGRhdG9zIHByb3BpbyBvIGFqZW5vIGEgbGEgVW5pdmVyc2lkYWQgeSBwYXJhIHF1ZSBlbiBsb3MgdMOpcm1pbm9zIGVzdGFibGVjaWRvcyBlbiBsYSBMZXkgMjMgZGUgMTk4MiwgTGV5IDQ0IGRlIDE5OTMsIERlY2lzacOzbiBBbmRpbmEgMzUxIGRlIDE5OTMsIERlY3JldG8gNDYwIGRlIDE5OTUgeSBkZW3DoXMgbm9ybWFzIGdlbmVyYWxlcyBzb2JyZSBsYSBtYXRlcmlhLCB1dGlsaWNlIGVuIHRvZGFzIHN1cyBmb3JtYXMsIGxvcyBkZXJlY2hvcyBwYXRyaW1vbmlhbGVzIGRlIHJlcHJvZHVjY2nDs24sIGNvbXVuaWNhY2nDs24gcMO6YmxpY2EsIHRyYW5zZm9ybWFjacOzbiB5IGRpc3RyaWJ1Y2nDs24gKGFscXVpbGVyLCBwcsOpc3RhbW8gcMO6YmxpY28gZSBpbXBvcnRhY2nDs24pIHF1ZSBtZSBjb3JyZXNwb25kZW4gY29tbyBjcmVhZG9yIGRlIGxhIG9icmEgb2JqZXRvIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8uPC9wPgo8cD5MYSBwcmVzZW50ZSBhdXRvcml6YWNpw7NuIHNlIGVtaXRlIGVuIGNhbGlkYWQgZGUgYXV0b3IgZGUgbGEgb2JyYSBvYmpldG8gZGVsIHByZXNlbnRlIGRvY3VtZW50byB5IG5vIGNvcnJlc3BvbmRlIGEgY2VzacOzbiBkZSBkZXJlY2hvcywgc2lubyBhIGxhIGF1dG9yaXphY2nDs24gZGUgdXNvIGFjYWTDqW1pY28gZGUgY29uZm9ybWlkYWQgY29uIGxvIGFudGVyaW9ybWVudGUgc2XDsWFsYWRvLiBMYSBwcmVzZW50ZSBhdXRvcml6YWNpw7NuIHNlIGhhY2UgZXh0ZW5zaXZhIG5vIHNvbG8gYSBsYXMgZmFjdWx0YWRlcyB5IGRlcmVjaG9zIGRlIHVzbyBzb2JyZSBsYSBvYnJhIGVuIGZvcm1hdG8gbyBzb3BvcnRlIG1hdGVyaWFsLCBzaW5vIHRhbWJpw6luIHBhcmEgZm9ybWF0byBlbGVjdHLDs25pY28sIHkgZW4gZ2VuZXJhbCBwYXJhIGN1YWxxdWllciBmb3JtYXRvIGNvbm9jaWRvIG8gcG9yIGNvbm9jZXIuPC9wPgo8cD5FbCBhdXRvciwgbWFuaWZpZXN0YSBxdWUgbGEgb2JyYSBvYmpldG8gZGUgbGEgcHJlc2VudGUgYXV0b3JpemFjacOzbiBlcyBvcmlnaW5hbCB5IGxhIHJlYWxpesOzIHNpbiB2aW9sYXIgbyB1c3VycGFyIGRlcmVjaG9zIGRlIGF1dG9yIGRlIHRlcmNlcm9zLCBwb3IgbG8gdGFudG8sIGxhIG9icmEgZXMgZGUgc3UgZXhjbHVzaXZhIGF1dG9yw61hIHkgdGllbmUgbGEgdGl0dWxhcmlkYWQgc29icmUgbGEgbWlzbWEuPC9wPgo8cD5FbiBjYXNvIGRlIHByZXNlbnRhcnNlIGN1YWxxdWllciByZWNsYW1hY2nDs24gbyBhY2Npw7NuIHBvciBwYXJ0ZSBkZSB1biB0ZXJjZXJvIGVuIGN1YW50byBhIGxvcyBkZXJlY2hvcyBkZSBhdXRvciBzb2JyZSBsYSBvYnJhIGVuIGN1ZXN0acOzbiwgZWwgYXV0b3IgYXN1bWlyw6EgdG9kYSBsYSByZXNwb25zYWJpbGlkYWQsIHkgc2FsZHLDoSBkZSBkZWZlbnNhIGRlIGxvcyBkZXJlY2hvcyBhcXXDrSBhdXRvcml6YWRvcywgcGFyYSB0b2RvcyBsb3MgZWZlY3RvcyBsYSBVbml2ZXJzaWRhZCBhY3TDumEgY29tbyB1biB0ZXJjZXJvIGRlIGJ1ZW5hIGZlLjwvcD4KPHA+U2kgdGllbmUgYWxndW5hIGR1ZGEgc29icmUgbGEgbGljZW5jaWEsIHBvciBmYXZvciwgY29udGFjdGUgY29uIGVsIDxhIGhyZWY9Im1haWx0bzpiaWJsaW90ZWNhQHVuaWFuZGVzLmVkdS5jbyIgdGFyZ2V0PSJfYmxhbmsiPkFkbWluaXN0cmFkb3IgZGVsIFNpc3RlbWEuPC9hPjwvcD4K |