No Cover Image

Journal article 406 views

Fine-tuning GPT-3 for legal rule classification

Davide Liga Orcid Logo, Livio Robaldo Orcid Logo

Computer Law & Security Review, Volume: 51, Start page: 105864

Swansea University Author: Livio Robaldo Orcid Logo

  • Accepted Manuscript under embargo until: 4th September 2024

Check full text

DOI (Published version): 10.1016/j.clsr.2023.105864

Abstract

In this paper, we propose a Legal Rule Classification (LRC) task using one of the most discussed language model in the field of Artificial Intelligence, namely GPT-3, a generative pretrained language model. We train and test the proposed LRC task on the GDPR encoded in LegalDocML (Palmirani and Vita...

Full description

Published in: Computer Law & Security Review
ISSN: 0267-3649
Published: Elsevier BV 2023
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa64410
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2023-09-04T20:00:43Z
last_indexed 2023-09-04T20:00:43Z
id cronfa64410
recordtype SURis
fullrecord <?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>64410</id><entry>2023-09-04</entry><title>Fine-tuning GPT-3 for legal rule classification</title><swanseaauthors><author><sid>b711cf9f3a7821ec52bd1e53b4f6cf9e</sid><ORCID>0000-0003-4713-8990</ORCID><firstname>Livio</firstname><surname>Robaldo</surname><name>Livio Robaldo</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-09-04</date><deptcode>LAWD</deptcode><abstract>In this paper, we propose a Legal Rule Classification (LRC) task using one of the most discussed language model in the field of Artificial Intelligence, namely GPT-3, a generative pretrained language model. We train and test the proposed LRC task on the GDPR encoded in LegalDocML (Palmirani and Vitali, 2011) and LegalRuleML (Athan et al., 2013), two widely used XML standards for the legal domain. We use the LegalDocML and LegalRuleML annotations provided in Robaldo et al. (2020) to fine-tuned GPT-3. While showing the ability of large language models (LLMs) to easily learn to classify legal and deontic rules even on small amount of data, we show that GPT-3 can significantly outperform previous experiments on the same task. Our work focused on a multiclass task, showing that GPT-3 is capable to recognize the difference between obligation rules, permission rules and constitutive rules with performances that overcome previous scores in LRC.</abstract><type>Journal Article</type><journal>Computer Law &amp; Security Review</journal><volume>51</volume><journalNumber/><paginationStart>105864</paginationStart><paginationEnd/><publisher>Elsevier BV</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>0267-3649</issnPrint><issnElectronic/><keywords>Rule classification, GPT-3, AI&amp;amp;Law</keywords><publishedDay>1</publishedDay><publishedMonth>11</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-11-01</publishedDate><doi>10.1016/j.clsr.2023.105864</doi><url>http://dx.doi.org/10.1016/j.clsr.2023.105864</url><notes/><college>COLLEGE NANME</college><department>Law</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>LAWD</DepartmentCode><institution>Swansea University</institution><apcterm/><funders/><projectreference/><lastEdited>2023-10-16T14:36:44.6032537</lastEdited><Created>2023-09-04T20:56:38.4664101</Created><path><level id="1">Faculty of Humanities and Social Sciences</level><level id="2">Hilary Rodham Clinton School of Law</level></path><authors><author><firstname>Davide</firstname><surname>Liga</surname><orcid>0000-0003-1124-0299</orcid><order>1</order></author><author><firstname>Livio</firstname><surname>Robaldo</surname><orcid>0000-0003-4713-8990</orcid><order>2</order></author></authors><documents><document><filename>Under embargo</filename><originalFilename>Under embargo</originalFilename><uploaded>2023-10-16T14:18:02.6449075</uploaded><type>Output</type><contentLength>2893350</contentLength><contentType>application/pdf</contentType><version>Accepted Manuscript</version><cronfaStatus>true</cronfaStatus><embargoDate>2024-09-04T00:00:00.0000000</embargoDate><documentNotes>Distributed under the terms of a Creative Commons CC-BY-NC-ND licence.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://doi.org/10.1016/j.clsr.2023.105864</licence></document></documents><OutputDurs/></rfc1807>
spelling v2 64410 2023-09-04 Fine-tuning GPT-3 for legal rule classification b711cf9f3a7821ec52bd1e53b4f6cf9e 0000-0003-4713-8990 Livio Robaldo Livio Robaldo true false 2023-09-04 LAWD In this paper, we propose a Legal Rule Classification (LRC) task using one of the most discussed language model in the field of Artificial Intelligence, namely GPT-3, a generative pretrained language model. We train and test the proposed LRC task on the GDPR encoded in LegalDocML (Palmirani and Vitali, 2011) and LegalRuleML (Athan et al., 2013), two widely used XML standards for the legal domain. We use the LegalDocML and LegalRuleML annotations provided in Robaldo et al. (2020) to fine-tuned GPT-3. While showing the ability of large language models (LLMs) to easily learn to classify legal and deontic rules even on small amount of data, we show that GPT-3 can significantly outperform previous experiments on the same task. Our work focused on a multiclass task, showing that GPT-3 is capable to recognize the difference between obligation rules, permission rules and constitutive rules with performances that overcome previous scores in LRC. Journal Article Computer Law & Security Review 51 105864 Elsevier BV 0267-3649 Rule classification, GPT-3, AI&amp;Law 1 11 2023 2023-11-01 10.1016/j.clsr.2023.105864 http://dx.doi.org/10.1016/j.clsr.2023.105864 COLLEGE NANME Law COLLEGE CODE LAWD Swansea University 2023-10-16T14:36:44.6032537 2023-09-04T20:56:38.4664101 Faculty of Humanities and Social Sciences Hilary Rodham Clinton School of Law Davide Liga 0000-0003-1124-0299 1 Livio Robaldo 0000-0003-4713-8990 2 Under embargo Under embargo 2023-10-16T14:18:02.6449075 Output 2893350 application/pdf Accepted Manuscript true 2024-09-04T00:00:00.0000000 Distributed under the terms of a Creative Commons CC-BY-NC-ND licence. true eng https://doi.org/10.1016/j.clsr.2023.105864
title Fine-tuning GPT-3 for legal rule classification
spellingShingle Fine-tuning GPT-3 for legal rule classification
Livio Robaldo
title_short Fine-tuning GPT-3 for legal rule classification
title_full Fine-tuning GPT-3 for legal rule classification
title_fullStr Fine-tuning GPT-3 for legal rule classification
title_full_unstemmed Fine-tuning GPT-3 for legal rule classification
title_sort Fine-tuning GPT-3 for legal rule classification
author_id_str_mv b711cf9f3a7821ec52bd1e53b4f6cf9e
author_id_fullname_str_mv b711cf9f3a7821ec52bd1e53b4f6cf9e_***_Livio Robaldo
author Livio Robaldo
author2 Davide Liga
Livio Robaldo
format Journal article
container_title Computer Law & Security Review
container_volume 51
container_start_page 105864
publishDate 2023
institution Swansea University
issn 0267-3649
doi_str_mv 10.1016/j.clsr.2023.105864
publisher Elsevier BV
college_str Faculty of Humanities and Social Sciences
hierarchytype
hierarchy_top_id facultyofhumanitiesandsocialsciences
hierarchy_top_title Faculty of Humanities and Social Sciences
hierarchy_parent_id facultyofhumanitiesandsocialsciences
hierarchy_parent_title Faculty of Humanities and Social Sciences
department_str Hilary Rodham Clinton School of Law{{{_:::_}}}Faculty of Humanities and Social Sciences{{{_:::_}}}Hilary Rodham Clinton School of Law
url http://dx.doi.org/10.1016/j.clsr.2023.105864
document_store_str 0
active_str 0
description In this paper, we propose a Legal Rule Classification (LRC) task using one of the most discussed language model in the field of Artificial Intelligence, namely GPT-3, a generative pretrained language model. We train and test the proposed LRC task on the GDPR encoded in LegalDocML (Palmirani and Vitali, 2011) and LegalRuleML (Athan et al., 2013), two widely used XML standards for the legal domain. We use the LegalDocML and LegalRuleML annotations provided in Robaldo et al. (2020) to fine-tuned GPT-3. While showing the ability of large language models (LLMs) to easily learn to classify legal and deontic rules even on small amount of data, we show that GPT-3 can significantly outperform previous experiments on the same task. Our work focused on a multiclass task, showing that GPT-3 is capable to recognize the difference between obligation rules, permission rules and constitutive rules with performances that overcome previous scores in LRC.
published_date 2023-11-01T14:36:46Z
_version_ 1779919388669902848
score 11.013082

Similar Items