https://www.mdu.se/

mdu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Requirements Ambiguity Detection and Explanation with LLMs: An Industrial Study
Mälardalen University, School of Innovation, Design and Engineering, Innovation and Product Realisation.ORCID iD: 0009-0006-8512-6412
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.ORCID iD: 0000-0001-6418-9971
Mälardalen University, School of Innovation, Design and Engineering, Embedded Systems.ORCID iD: 0000-0003-1688-6937
Show others and affiliations
2025 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Developing large-scale industrial systems requires high-quality requirements to avoid costly rework and project delays. However, linguistic ambiguities in natural language (NL) requirements have been a long-standing challenge, often introducing misinterpretations and inconsistencies that propagate throughout the development lifecycle. Such ambiguous NL requirements necessitate early detection and well-reasoned explanations to clarify and prevent further misunderstandings among stakeholders. While solutions have been developed to detect ambiguities in NL requirements, the advent of generative large language models (LLMs) offers new avenues for explanation-augmented requirements ambiguity detection. This paper empirically investigates LLMs for ambiguity detection and explanation in real-world industrial requirements by adopting an in-context learning paradigm. Our results from three industrial datasets show that LLMs achieve a 20.2% average performance increase in classifying ambiguous requirements when prompted with ten relevant in-context demonstrations (10-shot), compared to no demonstrations (0-shot). Additionally, we conducted human evaluations of the LLM-generated outputs with eight industry experts along four dimensions---naturalness, adequacy, usefulness and relevance---to gain practical insights. The results show an average rating of 3.84 out of 5 across evaluation criteria, indicating that the approach is effective in providing supporting explanations for requirement ambiguities.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2025. p. 620-631
Series
Proceedings - Conferense on Software Maintenance, ISSN 2576-3148
Keywords [en]
requirements classification, requirements ambiguity, large language models, in-context learning
National Category
Engineering and Technology Computer Sciences
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:mdh:diva-73154DOI: 10.1109/ICSME64153.2025.00063Scopus ID: 2-s2.0-105022457767ISBN: 979-8-3315-9587-6 (print)OAI: oai:DiVA.org:mdh-73154DiVA, id: diva2:1994011
Conference
41st IEEE International Conference on Software Maintenance and Evolution, Auckland, New Zealand, September 7–12, 2025
Available from: 2025-09-01 Created: 2025-09-01 Last updated: 2026-02-16Bibliographically approved
In thesis
1. AI-augmented Requirements Engineering for Industrial Systems
Open this publication in new window or tab >>AI-augmented Requirements Engineering for Industrial Systems
2025 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Engineering large-scale industrial systems requires an efficient Requirements Engineering (RE) process to manage the complexity resulting from continuous technological advancements. In manufacturing domains such as railways, the complexity of software-intensive systems is growing due to evolving standards, infrastructure specifications, and increasing customer expectations. Typically, the RE process begins with analyzing extensive tender documents from external customers to assess project feasibility. This analysis is critical, as the tender documents define the scope and the standards to which the system-to-be must comply. Once validated and agreed upon, the requirements are distributed among various subsystem teams for development and testing. During implementation, the evolving requirements are cross-referenced with existing technical documents to ensure consistency across project artifacts and prevent integration issues within subsystems. However, the reliance on manual efforts in performing these RE tasks makes the process labor-intensive and time-consuming, often leading to project scope creep in industrial settings.

This thesis empirically investigates Artificial Intelligence (AI), particularly Large Language Models (LLMs)-based solutions, to augment the RE process for realizing complex industrial systems. The proposed solutions aim to provide decision support to reduce the manual efforts typically required for (i) identifying requirements from other supporting information in tender documents, (ii) detecting ambiguous requirements and explaining them, (iii) allocating validated requirements to appropriate subsystem teams for development and (iv) addressing requirement-related queries during the development and release phases of the project. Consequently, this research contributes to enhancing requirements management in complex industrial systems by enabling more efficient and informed decision-making.

Abstract [sv]

Att konstruera storskaliga industriella system kräver en effektiv Requirements Engineering (RE)-process för att hantera den komplexitet som följer av kontinuerliga tekniska framsteg. Inom tillverkningsdomäner som järnväg ökar komplexiteten i mjukvaruintensiva system på grund av standarder som förändras, infrastrukturspecifikationer och ökade kundförväntningar. RE-processen inleds vanligen med att omfattande anbudsunderlag från externa beställare analyseras för att bedöma projektets genomförbarhet. Denna analys är avgörande eftersom anbudsunderlaget fastställer omfattningen och de standarder som det framtida systemet måste uppfylla. När kraven har validerats och godkänts fördelas de till olika delsystemteam för utveckling och test. Under implementeringen korsrefereras de föränderliga kraven med befintlig teknisk dokumentation för att säkerställa konsekvens emellan artefakter och förhindra integrationsproblem mellan delsystem. Det manuella arbete som krävs för dessa RE-uppgifter gör dock processen arbets- och tidskrävande, vilket ofta leder till 'scope creep' i industriella projekt.

Denna avhandling undersöker empiriskt AI-, särskilt LLMs-baserade, lösningar för att stärka RE-processen vid realisering av komplexa industriella system. De föreslagna lösningarna syftar till att ge beslutsstöd för att minska det manuella arbete som normalt krävs för (i) att identifiera krav bland övrig information i anbudsunderlag, (ii) att upptäcka tvetydiga krav och förklara dem, (iii) att fördela validerade krav till lämpliga delsystemteam för utveckling och (iv) att besvara kravsrelaterade frågor under projektets utvecklings- och releasefaser. Därmed bidrar forskningen till förbättrad kravhantering i komplexa industriella system genom att möjliggöra effektivare och mer välgrundade beslut.

Place, publisher, year, edition, pages
Västerås: Mälardalens universitet, 2025. p. 172
Series
Mälardalen University Press Licentiate Theses, ISSN 1651-9256 ; 378
National Category
Software Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:mdh:diva-73156 (URN)978-91-7485-720-7 (ISBN)
Presentation
2025-10-16, Case, Mälardalens universitet och digitalt., Västerås, 13:15 (English)
Opponent
Supervisors
Funder
Knowledge FoundationEuropean CommissionVinnova
Available from: 2025-09-02 Created: 2025-09-01 Last updated: 2025-10-10Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Bashir, SarmadAbbas, MuhammadStrandberg, Per ErikHaider, ZulqarnainSaadatmand, MehrdadBohlin, Markus

Search in DiVA

By author/editor
Bashir, SarmadFerrari, AlessioAbbas, MuhammadStrandberg, Per ErikHaider, ZulqarnainSaadatmand, MehrdadBohlin, Markus
By organisation
Innovation and Product RealisationEmbedded Systems
Engineering and TechnologyComputer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 1192 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf