No Cover Image

Journal article 743 views 58 downloads

Fine-tuning GPT-3 for legal rule classification

Davide Liga Orcid Logo, Livio Robaldo Orcid Logo

Computer Law & Security Review, Volume: 51, Start page: 105864

Swansea University Author: Livio Robaldo Orcid Logo

  • 64410.AAM.pdf

    PDF | Accepted Manuscript

    Distributed under the terms of a Creative Commons CC-BY-NC-ND licence.

    Download (2.76MB)

Check full text

DOI (Published version): 10.1016/j.clsr.2023.105864

Abstract

In this paper, we propose a Legal Rule Classification (LRC) task using one of the most discussed language model in the field of Artificial Intelligence, namely GPT-3, a generative pretrained language model. We train and test the proposed LRC task on the GDPR encoded in LegalDocML (Palmirani and Vita...

Full description

Published in: Computer Law & Security Review
ISSN: 0267-3649
Published: Elsevier BV 2023
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa64410
first_indexed 2023-09-04T20:00:43Z
last_indexed 2024-11-25T14:13:59Z
id cronfa64410
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2024-09-05T13:46:29.8152915</datestamp><bib-version>v2</bib-version><id>64410</id><entry>2023-09-04</entry><title>Fine-tuning GPT-3 for legal rule classification</title><swanseaauthors><author><sid>b711cf9f3a7821ec52bd1e53b4f6cf9e</sid><ORCID>0000-0003-4713-8990</ORCID><firstname>Livio</firstname><surname>Robaldo</surname><name>Livio Robaldo</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-09-04</date><deptcode>HRCL</deptcode><abstract>In this paper, we propose a Legal Rule Classification (LRC) task using one of the most discussed language model in the field of Artificial Intelligence, namely GPT-3, a generative pretrained language model. We train and test the proposed LRC task on the GDPR encoded in LegalDocML (Palmirani and Vitali, 2011) and LegalRuleML (Athan et al., 2013), two widely used XML standards for the legal domain. We use the LegalDocML and LegalRuleML annotations provided in Robaldo et al. (2020) to fine-tuned GPT-3. While showing the ability of large language models (LLMs) to easily learn to classify legal and deontic rules even on small amount of data, we show that GPT-3 can significantly outperform previous experiments on the same task. Our work focused on a multiclass task, showing that GPT-3 is capable to recognize the difference between obligation rules, permission rules and constitutive rules with performances that overcome previous scores in LRC.</abstract><type>Journal Article</type><journal>Computer Law &amp; Security Review</journal><volume>51</volume><journalNumber/><paginationStart>105864</paginationStart><paginationEnd/><publisher>Elsevier BV</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>0267-3649</issnPrint><issnElectronic/><keywords>Rule classification, GPT-3, AI&amp;amp;Law</keywords><publishedDay>1</publishedDay><publishedMonth>11</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-11-01</publishedDate><doi>10.1016/j.clsr.2023.105864</doi><url>http://dx.doi.org/10.1016/j.clsr.2023.105864</url><notes/><college>COLLEGE NANME</college><department>Hillary Rodham Clinton Law School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>HRCL</DepartmentCode><institution>Swansea University</institution><apcterm/><funders/><projectreference/><lastEdited>2024-09-05T13:46:29.8152915</lastEdited><Created>2023-09-04T20:56:38.4664101</Created><path><level id="1">Faculty of Humanities and Social Sciences</level><level id="2">Hilary Rodham Clinton School of Law</level></path><authors><author><firstname>Davide</firstname><surname>Liga</surname><orcid>0000-0003-1124-0299</orcid><order>1</order></author><author><firstname>Livio</firstname><surname>Robaldo</surname><orcid>0000-0003-4713-8990</orcid><order>2</order></author></authors><documents><document><filename>64410__28795__5325de6920c3470ab23be2a97b1077c0.pdf</filename><originalFilename>64410.AAM.pdf</originalFilename><uploaded>2023-10-16T14:18:02.6449075</uploaded><type>Output</type><contentLength>2893350</contentLength><contentType>application/pdf</contentType><version>Accepted Manuscript</version><cronfaStatus>true</cronfaStatus><embargoDate>2024-09-04T00:00:00.0000000</embargoDate><documentNotes>Distributed under the terms of a Creative Commons CC-BY-NC-ND licence.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://doi.org/10.1016/j.clsr.2023.105864</licence></document></documents><OutputDurs/></rfc1807>
spelling 2024-09-05T13:46:29.8152915 v2 64410 2023-09-04 Fine-tuning GPT-3 for legal rule classification b711cf9f3a7821ec52bd1e53b4f6cf9e 0000-0003-4713-8990 Livio Robaldo Livio Robaldo true false 2023-09-04 HRCL In this paper, we propose a Legal Rule Classification (LRC) task using one of the most discussed language model in the field of Artificial Intelligence, namely GPT-3, a generative pretrained language model. We train and test the proposed LRC task on the GDPR encoded in LegalDocML (Palmirani and Vitali, 2011) and LegalRuleML (Athan et al., 2013), two widely used XML standards for the legal domain. We use the LegalDocML and LegalRuleML annotations provided in Robaldo et al. (2020) to fine-tuned GPT-3. While showing the ability of large language models (LLMs) to easily learn to classify legal and deontic rules even on small amount of data, we show that GPT-3 can significantly outperform previous experiments on the same task. Our work focused on a multiclass task, showing that GPT-3 is capable to recognize the difference between obligation rules, permission rules and constitutive rules with performances that overcome previous scores in LRC. Journal Article Computer Law & Security Review 51 105864 Elsevier BV 0267-3649 Rule classification, GPT-3, AI&amp;Law 1 11 2023 2023-11-01 10.1016/j.clsr.2023.105864 http://dx.doi.org/10.1016/j.clsr.2023.105864 COLLEGE NANME Hillary Rodham Clinton Law School COLLEGE CODE HRCL Swansea University 2024-09-05T13:46:29.8152915 2023-09-04T20:56:38.4664101 Faculty of Humanities and Social Sciences Hilary Rodham Clinton School of Law Davide Liga 0000-0003-1124-0299 1 Livio Robaldo 0000-0003-4713-8990 2 64410__28795__5325de6920c3470ab23be2a97b1077c0.pdf 64410.AAM.pdf 2023-10-16T14:18:02.6449075 Output 2893350 application/pdf Accepted Manuscript true 2024-09-04T00:00:00.0000000 Distributed under the terms of a Creative Commons CC-BY-NC-ND licence. true eng https://doi.org/10.1016/j.clsr.2023.105864
title Fine-tuning GPT-3 for legal rule classification
spellingShingle Fine-tuning GPT-3 for legal rule classification
Livio Robaldo
title_short Fine-tuning GPT-3 for legal rule classification
title_full Fine-tuning GPT-3 for legal rule classification
title_fullStr Fine-tuning GPT-3 for legal rule classification
title_full_unstemmed Fine-tuning GPT-3 for legal rule classification
title_sort Fine-tuning GPT-3 for legal rule classification
author_id_str_mv b711cf9f3a7821ec52bd1e53b4f6cf9e
author_id_fullname_str_mv b711cf9f3a7821ec52bd1e53b4f6cf9e_***_Livio Robaldo
author Livio Robaldo
author2 Davide Liga
Livio Robaldo
format Journal article
container_title Computer Law & Security Review
container_volume 51
container_start_page 105864
publishDate 2023
institution Swansea University
issn 0267-3649
doi_str_mv 10.1016/j.clsr.2023.105864
publisher Elsevier BV
college_str Faculty of Humanities and Social Sciences
hierarchytype
hierarchy_top_id facultyofhumanitiesandsocialsciences
hierarchy_top_title Faculty of Humanities and Social Sciences
hierarchy_parent_id facultyofhumanitiesandsocialsciences
hierarchy_parent_title Faculty of Humanities and Social Sciences
department_str Hilary Rodham Clinton School of Law{{{_:::_}}}Faculty of Humanities and Social Sciences{{{_:::_}}}Hilary Rodham Clinton School of Law
url http://dx.doi.org/10.1016/j.clsr.2023.105864
document_store_str 1
active_str 0
description In this paper, we propose a Legal Rule Classification (LRC) task using one of the most discussed language model in the field of Artificial Intelligence, namely GPT-3, a generative pretrained language model. We train and test the proposed LRC task on the GDPR encoded in LegalDocML (Palmirani and Vitali, 2011) and LegalRuleML (Athan et al., 2013), two widely used XML standards for the legal domain. We use the LegalDocML and LegalRuleML annotations provided in Robaldo et al. (2020) to fine-tuned GPT-3. While showing the ability of large language models (LLMs) to easily learn to classify legal and deontic rules even on small amount of data, we show that GPT-3 can significantly outperform previous experiments on the same task. Our work focused on a multiclass task, showing that GPT-3 is capable to recognize the difference between obligation rules, permission rules and constitutive rules with performances that overcome previous scores in LRC.
published_date 2023-11-01T14:33:32Z
_version_ 1821416371524730880
score 11.247077

Similar Items