Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions _bibliography/references.bib
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,35 @@ @article{zhao_natural_2021
keywords = {systematic review, systematic mapping study, natural language processing ({NLP}), Requirements engineering ({RE}), software engineering ({SE})},
}

@article{arora_advancing_2023,
title = {Advancing Requirements Engineering through Generative {AI}: Assessing the Role of {LLMs}},
url = {https://arxiv.org/abs/2310.13976},
doi = {10.48550/arXiv.2310.13976},
abstract = {Requirements Engineering ({RE}) is a critical phase in software development including the elicitation, analysis, specification, and validation of software requirements. Despite the importance of {RE}, it remains a challenging process due to the complexities of communication, uncertainty in the early stages and inadequate automation support. In recent years, large-language models ({LLMs}) have shown significant promise in diverse domains, including natural language processing, code generation, and program understanding. This chapter explores the potential of {LLMs} in driving {RE} processes, aiming to improve the efficiency and accuracy of requirements-related tasks. We propose key directions and {SWOT} analysis for research and development in using {LLMs} for {RE}, focusing on the potential for requirements elicitation, analysis, specification, and validation. We further present the results from a preliminary evaluation, in this context.},
author = {Arora, Chetan and Grundy, John and Abdelrazek, Mohamed},
urldate = {2024-01-15},
date = {2023-11-01},
eprinttype = {arxiv},
eprint = {2310.13976v2},
keywords = {Requirements Engineering, Generative {AI}, Large Language Models ({LLMs}), Natural Language Processing, Software Engineering},
}

@article{marques_using_2024,
title = {Using {ChatGPT} in Software Requirements Engineering: A Comprehensive Review},
volume = {16},
issn = {1999-5903},
url = {https://www.mdpi.com/1999-5903/16/6/180},
doi = {10.3390/fi16060180},
abstract = {Large language models ({LLMs}) have had a significant impact on several domains, including software engineering. However, a comprehensive understanding of {LLMs}' use, impact, and potential limitations in software engineering is still emerging and remains in its early stages. This paper analyzes the role of large language models ({LLMs}), such as {ChatGPT}-3.5, in software requirements engineering, a critical area in software engineering experiencing rapid advances due to artificial intelligence ({AI}). By analyzing several studies, we systematically evaluate the integration of {ChatGPT} into software requirements engineering, focusing on its benefits, challenges, and ethical considerations. This evaluation is based on a comparative analysis that highlights {ChatGPT}'s efficiency in eliciting requirements, accuracy in capturing user needs, potential to improve communication among stakeholders, and impact on the responsibilities of requirements engineers. The selected studies were analyzed for their insights into the effectiveness of {ChatGPT}, the importance of human feedback, prompt engineering techniques, technological limitations, and future research directions in using {LLMs} in software requirements engineering. This comprehensive analysis aims to provide a differentiated perspective on how {ChatGPT} can reshape software requirements engineering practices and provides strategic recommendations for leveraging {ChatGPT} to effectively improve the software requirements engineering process.},
number = {6},
journaltitle = {Future Internet},
author = {Marques, Nuno and Silva, Rodrigo Rocha and Bernardino, Jorge},
urldate = {2024-05-21},
date = {2024-05-21},
langid = {english},
keywords = {{ChatGPT}, {LLMs}, software engineering, software requirements, generative {AI}},
}

@article{alzoubi_empirical_2016,
title = {Empirical studies of geographically distributed agile development communication challenges: A systematic review},
volume = {53},
Expand Down
133 changes: 133 additions & 0 deletions content/needs/reqs/llms.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
---
parent: Requirements
title: LLMs for Requirements Management
nav_order: 8
layout: default
---

# LLMs for Requirements Management

Large Language Models (LLMs) such as ChatGPT, DeepSeek, and other generative AI systems have shown significant potential in supporting various Requirements Engineering (RE) tasks. This emerging area explores how AI can assist requirements engineers throughout the entire RE lifecycle.

---

## Overview

Most requirements are specified using natural language, making LLMs particularly well-suited for requirements-related tasks. LLMs can process and generate human language by learning patterns from vast amounts of text data, enabling them to assist with requirements elicitation, specification, analysis, and validation.

**Key promise:** LLMs can help automate and streamline RE tasks, address communication challenges, and improve the efficiency and quality of requirements work.

---

## Applications Across RE Stages

### Requirements Elicitation

LLMs can assist in:
- **Domain analysis**: Rapidly processing domain-specific literature to provide foundational knowledge
- **Stakeholder communication**: Translating technical jargon into plain language and supporting multilingual stakeholders
- **Identifying unknowns**: Analyzing documentation to highlight areas of ambiguity or uncertainty
- **Generating requirements**: Creating preliminary requirements based on initial inputs and stakeholder personas

**Example use case:** Using LLMs to simulate stakeholder interviews by providing personas (e.g., "act as a 65-year-old retired patient with type-2 diabetes") to elicit requirements from different perspectives.

---

### Requirements Specification

LLMs can help with:
- **Formatting**: Converting unstructured requirements into structured formats (e.g., EARS template, user stories)
- **Classification**: Categorizing requirements into functional and non-functional requirements
- **Compliance checking**: Cross-referencing requirements against standards and regulatory guidelines
- **Generating artifacts**: Creating glossaries, examples, rationale, and user personas

**Example use case:** Converting informal requirements like "patients should not receive notifications when busy" into structured EARS format: "When patient is driving, ActApp shall not send notifications."

---

### Requirements Analysis

LLMs can support:
- **Quality evaluation**: Automatically assessing requirements for ambiguities, inconsistencies, or incompleteness
- **Risk identification**: Identifying potential risks and suggesting countermeasures
- **Conflict resolution**: Facilitating negotiation by simulating multiple stakeholder perspectives
- **Change impact analysis**: Providing real-time feedback during requirements refinement

**Example use case:** Using multiple LLM agents representing different stakeholders (e.g., patient vs. software architect) to negotiate and prioritize requirements, ensuring all perspectives are considered.

---

### Requirements Validation

LLMs can assist with:
- **Simulating stakeholder perspectives**: Anticipating potential misinterpretations or misalignments
- **Traceability**: Assisting in linking requirements to design documents and regulatory codes
- **Acceptance criteria**: Formulating clear and precise acceptance criteria based on documented requirements
- **Test scenarios**: Proposing test scenarios for comprehensive validation

**Example use case:** Generating acceptance criteria for a requirement: "R1.1-AC1: Accurately detect when the user has been sitting for 60 continuous minutes. R1.1-AC2: Notifications can be toggled on or off by user."

---

## Key Benefits

Based on comprehensive reviews of recent research, LLMs offer several key benefits in requirements engineering:

- **Automation**: Streamlining repetitive tasks like formatting, classification, and documentation
- **Consistency**: Providing uniform analysis and reducing human bias or fatigue
- **Efficiency and Accuracy**: Processing large volumes of requirements data quickly, with quality comparable to human-generated requirements
- **Enhanced Productivity**: Automating documentation processes allows developers to focus on more creative problem-solving tasks
- **Brainstorming and Idea Exploration**: Generating diverse ideas and suggestions, stimulating creativity and exploring different possibilities from multiple perspectives
- **Continuous Learning**: Adapting and improving through interactions and feedback, expanding knowledge base over time
- **Minimizing Human Error**: Reducing errors associated with manual transcription and interpretation, ensuring consistent documentation
- **Cost Savings**: Streamlining requirements gathering and analysis processes, reducing development time and manual effort
- **Accessibility**: Translating complex technical language for diverse stakeholders
- **Completeness**: Helping identify missing requirements or overlooked aspects

---

## Challenges and Limitations

Research has identified several critical challenges and limitations when using LLMs in requirements engineering:

- **Domain expertise**: LLMs may lack deep domain knowledge that human experts possess
- **Context limitations**: Limited context window may cause loss of information in large documents
- **Ambiguity handling**: Struggling with inherently ambiguous or conflicting requirements, leading to reasoning errors
- **Over-automation risk**: Risk of sidelining human expertise and missing critical nuances
- **Data security and privacy**: Concerns about sharing sensitive requirements with public LLM services; susceptibility to adversarial attacks (prompt injection, jailbreak attacks)
- **Bias**: Potential for introducing or perpetuating biases from training data, user interactions, algorithms, or contextual factors
- **Information hallucination**: LLMs may generate incorrect or purely fictional information when lacking sufficient knowledge
- **Lack of explainability and transparency**: Difficulty understanding how LLMs arrive at their decisions, which can undermine developer confidence
- **Over-reliance**: Temptation to rely too heavily on generated text without sufficient verification, potentially overlooking contextual nuances
- **Experience dependency**: Effective use requires experience in prompt engineering and RE practices

---

## Practical Considerations

**Prompt engineering** is crucial for effective LLM use in RE. Well-designed prompts should:
- Provide clear context about the project and domain
- Specify the expected output format (e.g., user stories, EARS template)
- Include examples (few-shot prompting) when needed
- Use appropriate prompt patterns (e.g., persona-based, chain-of-thought)

**Human-in-the-loop** approach: LLMs should complement, not replace, human requirements engineers. Domain experts bring essential intuition, cultural awareness, and nuanced understanding that LLMs cannot fully replicate.

---

## Research Evidence

Preliminary evaluations show that experienced requirements engineers achieve better results when using LLMs for elicitation tasks (e.g., precision rates of 53-82% vs. 15-29% for less experienced users). This highlights the importance of RE expertise in effectively leveraging LLM capabilities.

---

## References

1. C. Arora, J. Grundy, and M. Abdelrazek, "Advancing Requirements Engineering through Generative AI: Assessing the Role of LLMs," arXiv preprint arXiv:2310.13976, Nov. 2023. [arXiv](https://arxiv.org/abs/2310.13976)
2. N. Marques, R. R. Silva, and J. Bernardino, "Using ChatGPT in Software Requirements Engineering: A Comprehensive Review," Future Internet, vol. 16, no. 6, p. 180, May 2024, doi: 10.3390/fi16060180. [MDPI](https://www.mdpi.com/1999-5903/16/6/180)
3. L. Zhao et al., "Natural Language Processing for Requirements Engineering: A Systematic Mapping Study," ACM Computing Surveys, vol. 54, no. 3, pp. 55:1-55:41, Apr. 2021, doi: 10.1145/3444689.

---

{: .highlight }
**Disclaimer:** AI is used for text summarization, explaining and formatting. Authors have verified all facts and claims. In case of an error, feel free to file an issue.