Conference Paper/Proceeding/Abstract 301 views 12 downloads
An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs
,
Nicholas Micallef ,
Julian Hough
Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), Pages: 10390 - 10404
Swansea University Authors:
Deshan Sumanathilaka , Nicholas Micallef , Julian Hough
-
PDF | Version of Record
Licenced under CC-BY-NC-4.0, the Creative Commons Attribution-NonCommercial 4.0 International License.
Download (389.07KB)
DOI (Published version): 10.63317/3oun2fvikwt5
Abstract
Word Sense Disambiguation (WSD) remains a key challenge in Natural Language Processing (NLP), especially when dealing with rare or domain-specific senses that are often misinterpreted. While modern high-parameter Large Language Models (LLMs) such as GPT-4-Turbo have shown state-of-the-art WSD perfor...
| Published in: | Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026) |
|---|---|
| ISBN: | 978-2-493814-49-4 |
| ISSN: | 2522-2686 |
| Published: |
European Language Resources Association (ELRA)
2026
|
| Online Access: |
Check full text
|
| URI: | https://cronfa.swan.ac.uk/Record/cronfa71528 |
| Abstract: |
Word Sense Disambiguation (WSD) remains a key challenge in Natural Language Processing (NLP), especially when dealing with rare or domain-specific senses that are often misinterpreted. While modern high-parameter Large Language Models (LLMs) such as GPT-4-Turbo have shown state-of-the-art WSD performance, their computational and energy demands limit scalability. This study investigates whether low-parameter LLMs (<4B parameters) can achieve comparable results through fine-tuning strategies that emphasize reasoning-driven sense identification. Using the FEWS dataset augmented with semi-automated, rationale-rich annotations, we fine-tune eight small-scale open-source LLMs (e.g. Gemma and Qwen). Our results reveal that Chain-of-Thought (CoT)-based reasoning combined with neighbour-word analysis achieves performance comparable to GPT-4-Turbo in zero-shot settings. Importantly, Gemma-3-4B and Qwen-3-4B models consistently outperform all medium-parameter baselines and state-of-the-art models on FEWS, with robust generalization to unseen senses. Furthermore, evaluation on the unseen "Fool Me If You Can” dataset confirms strong cross-domain adaptability without task-specific fine-tuning. This work demonstrates that with carefully crafted reasoning-centric fine-tuning, low-parameter LLMs can deliver accurate WSD while substantially reducing computational and energy demands. |
|---|---|
| Keywords: |
Word Sense Disambiguation, Low-parameter LLMs, Reasoning-driven Fine-tuning |
| College: |
Faculty of Science and Engineering |
| Funders: |
We acknowledge the support of the Super computing Wales project, which is part-funded by the European Regional Development Fund (ERDF) via Welsh Government. Hough’s work is supported by the EPSRC grant EP/X009343/1 ‘FLUIDITY’. |
| Start Page: |
10390 |
| End Page: |
10404 |