Open Dataset for the Study "Systematic Mapping Study on Requirements Engineering for Regulatory Compliance of Software Systems"

Open Dataset for Study "Systematic Mapping Study on Requirements Engineering for Regulatory Compliance of Software Systems"

Authors: Oleksandr Kosenkov

, Parisa Elahidoost

, Tony Gorschek

, Jannik Fischbach

, Daniel Mendez

, Michael Unterkalmsteiner

, Davide Fucci

, Rahul Mohanani

Published in "Information and Software Technology" (IST) Journal. Publication available at arXiv (preprint) and on Elsevier website. Dataset is also available on Zenodo (DOI: 10.5281/zenodo.13999201).

Context: As the diversity and complexity of regulations affecting Software-Intensive Products and Services (SIPS) is increasing, software engineers need to address the growing regulatory scrutiny. We argue that as with any other non-negotiable requirements, SIPS compliance should be addressed early in SIPS engineering - i.e., during requirements engineering (RE).

Objectives: In the conditions of the expanding regulatory landscape, existing research offers scattered insights into regulatory compliance of SIPS. This study addresses the pressing need for a structured overview of the state of the art in software RE and its contribution to regulatory compliance of SIPS.

Method: We conducted a systematic mapping study to provide an overview of the current state of research regarding challenges, principles, and practices for regulatory compliance of SIPS related to RE. We focused on the role of RE and its contribution to other SIPS lifecycle process areas. We retrieved \added{6914} studies published from 2017 (January 1) until 2023 (December 31) from four academic databases, which we filtered down to 280 relevant primary studies.

Results: We identified and categorized the RE-related challenges in regulatory compliance of SIPS and their potential connection to six types of principles and practices addressing challenges. We found that about 13\% of the primary studies considered the involvement of legal experts in developing principles and practices. About 20\% of primary studies considered RE in connection to other process areas. Most primary studies focused on a few popular regulation fields (privacy, safety) and application domains (healthcare, avionics). Our results suggest that there can be differences in terms of challenges and involvement of stakeholders across different fields of regulation.

Conclusion: Our findings highlight the need for an in-depth investigation of stakeholders' roles, relationships between process areas, and specific challenges for distinct regulatory fields to guide research and practice.

RQ1: What are the reported challenges in regulatory compliance of SIPS?

RQ2: What is needed to address the challenges to the regulatory compliance of SIPS, and what principles and practices are used?

RQ3: Which stakeholders are involved in developing, applying and/or validating principles and practices for regulatory compliance of SIPS?

RQ4: What are the main software process areas (PAs) involved in enabling the regulatory compliance of SIPS?

RQ5: Which regulations and domains of application are the most prevalent in regulatory compliance of SIPS?

RQ1: Any mention of a challenge was counted. Each category of challenge was only counted once for the primary study, even if that category was mentioned multiple times.

RQ3: Any mention of stakeholder was counted. Each type of stakeholder was counted only once for the primary study, even if that type was mentioned multiple times. Stakeholder-to-challenge mapping: each category of stakeholder identified in a primary study was mapped to all the categories of challenges mentioned in that study. Stakeholder-to-fields of regulation mapping: each category of stakeholder identified in a primary study was mapped to all the fields of regulation mentioned in that study. Stakeholder-to-principles and practices mapping: each category of stakeholder identified in a primary study was mapped to all the types of principles and practices mentioned in that study.

RQ4: Table 7.1: Any mention of RE processes (e.g., requirements elicitation (REE)) was counted as either standalone RE or RE in combination with other SDLC process areas. Table 7.2: Only mentions of RE processes were counted, regardless of whether they were considered independently or in conjunction with other SDLC process areas.

Challenges to the regulatory compliance of SIPS are general types of SE issues that emerge in achieving the regulatory compliance of SIPS.

Principles and practices (PPs) for regulatory compliance of SIPS are any means employed to implement regulatory compliance of SIPS and to tackle the related challenges. These includes but is not limited to SE methods, tools, frameworks, solutions, and models.

Automation refers to the full or partial replacement of a function previously carried out by the human operator (Parasuraman et al. 2000). The levels of automation are as follows (Parasuraman et al. 2000):

information acquisition: automation in acquiring information applies to sensing and recording input data.
information analysis: automation in information analysis incorporates cognitive functions, such as working memory and inferential processes.
decision and action selection: The third stage, which involves decision and action selection, requires choosing from various alternatives. Automating this stage encompasses varying degrees of enhancing or substituting human decision-making with machine-based decision-making processes.
action implementation: The last stage, action implementation, pertains to the actual execution of the chosen action. Automating this stage involves varying degrees of machine-based execution of the selected action, typically substituting human manual or vocal execution.

Stakeholder types are as follows: software engineering roles, legal or compliance experts, researchers, other experts, and other stakeholders. We consider that the life cycle of PPs comprises three core stages: development, application, and validation.

We identify the following process areas (PAs) in the SIPS life cycle based on SWEBOK and ISO/IEC 12207 and deduce their potential contribution to regulatory compliance of SIPS:

SIPS Requirements Engineering (RE)---systematic handling of requirements to SIPS that derive from regulations. This main process area includes the following processes: requirements elicitation, requirements analysis and modeling (REA/M), requirements specification (RES), requirements verification and validation (REVV), and requirements management (REM). Regulatory RE is an area of requirements engineering practice and research that contributes to the compliance of software systems by processing requirements derived from regulations for SE purposes.
SIPS design (SD) - the process of defining the internal structure (e.g., architecture, components, interfaces) of SIPS in a way that will enable their regulatory compliance (e.g., assure verifiability, address future evolution of regulations).
SIPS development (SDev) refers to the detailed creation of working SIPS through coding, verification, unit testing, integration testing, and debugging. SIPS development process areas are related to implementing requirements derived from regulations.
SIPS quality assurance (SQA) consists of the dynamic verification that SIPS provide expected behaviors on a set of test cases addressing the regulators' perspective on compliance by verifying the implementation of all requirements originating from regulations (e.g., \cite{corriveau2014requirements}) and also by using tools and methods applied by regulators for compliance verification.
In our study, we define a SIPS deployment as a set of activities directed towards generating executable and testable SIPS components, combining related components with a single deployable artefact and putting SIPS into operation. With the advancement of DevOps and automation practices, it becomes essential to ensure regulatory compliance of the SIPS deployment process and regulatory compliance of the deployed SIPS.
SIPS maintenance (SM) is the totality of activities required to provide cost-effective support to software to ensure that SIPS remains compliant throughout the software and regulatory evolution.

Field of regulation (like the concept of "field of law") is a group of social relations that are addressed by the regulation (e.g., GDPR belongs to the personal data protection field of regulation). In our study we have identified the following fields of regulation: Accessibility, AI/ML (capturing any regulations applicable to AI/ML-based software systems), Business (capturing regulations applicable to both business processes and/or enterprise information systems across different industries), Privacy, Privacy\&Security (identified in case same regulation considers both Privacy and Security simultaneously), Quality, Safety, Security, Traffic law, User rights (capturing regulations applicable to SIPS users only, e.g., human rights, patient rights).

Domain of application - operational domain in which SIPS are engineered, and PPs for the regulatory compliance of SIPS are applied. In our study we have defined the following domains of application: Automotive, Avionics, Cloud computing, E-commerce, Education, Energy (including nuclear energy, electricity distribution), Enterprise (capturing wide range industries (e.g., retail, food industries) and activities characteristic or enterprises (e.g., marketing, taxation)), Finances, Government, Healthcare, IoT, Manufacturing, Media, Metrology, Military, Smart home/city, Software development (capturing primary study which focused on SIPS product and outsourcing companies, not belonging to specific industry), Telecommunications, Transport.

Scoring rubrics for evaluating rigor and relevance are described according to Ivarsson & Gorschek, T. (2011)

Data

Select columns you would like to display

(?):

Columns are hidden in the webview and will be visible when you export data

ID - Title - Venue - Year - Authors - Affiliations - Affiliation type - Abstract - RQ1: Challenges to compliance - RQ1: Categories of challenges - RQ2: Principles and practices (PPs) - RQ2: PPs type - RQ2: Automation - RQ2: Automation type - RQ3: Involved stakeholders - RQ3: Stakeholders types - RQ3: Involvement phase - RQ4: SDLC process areas (PAs) - RQ5: Regulations considered - RQ5: Field of regulation - RQ5: Domain of application - Context described - Study design described - Validity discussed - Subjects - Context - Scale - Research method:

ID	Title	Venue	Year	Authors	Affiliations	Affiliation type	Abstract	RQ1: Challenges to compliance	RQ1: Challenges categories	RQ2: Principles and practices	RQ2: PPs type	RQ2.1: Automation	RQ2.1: Automation type	RQ3: Involved stakeholders	RQ3: Types of stakehodlers	RQ3: Phase in which stakehoders invovled	RQ4: MPAs	RQ5: Regulations considered	RQ5: Field of regualtion	RQ5: Domain of applicaiton	Context	Study design	Validity	Subjects	Context	Scale	Research method
ID	Title	Venue	Year	Authors	Affiliations	Affiliation type	Abstract	RQ1: Challenges to compliance	RQ1: Challenges categories	RQ2: Principles and practices	RQ2: PPs type	RQ2.1: Automation	RQ2.1: Automation type	RQ3: Involved stakeholders	RQ3: Types of stakehodlers	RQ3: Phase in which stakehoders invovled	RQ4: MPAs	RQ5: Regulations considered	RQ5: Field of regualtion	RQ5: Domain of applicaiton	Context	Study design	Validity	Subjects	Context	Scale	Research method