https://www.mdu.se/

Passive testing is an approach to verify system behavior by observing logs from normal operation, without actively injecting test stimuli. This paper presents an industrial case study of applying passive testing in the domain of vehicular embedded systems, utilizing two specialized tools: Timed Easy Approach to Requirements Syntax (T-EARS) for specifying temporal requirements, and Napkin Studio for evaluating these requirements against real system execution logs. We collaborated with Volvo Construction Equipment (VCE) to translate a set of natural language requirements into structured T-EARS specifications. Then we used Napkin Studio to test these requirements against recorded machine log data passively. We evaluate the feasibility of this approach, the extent to which it can detect requirement violations or injected faults, and the perceptions of industry stakeholders regarding the adoption of such passive tests in their verification process. The results show that a majority of functional requirements can be expressed as Guarded Assertions (GAs) and validated on logs, uncovering specific issues. Stakeholders found the method promising for improving test coverage and efficiency, although integration challenges (e.g., log signal inconsistencies and tool usability issues) were noted. Overall, this work provides empirical evidence that passive testing with T-EARS and Napkin Studio can complement traditional hardware-in-the-loop testing, offering a scalable and non-intrusive verification approach in developing vehicular systems.

Model-based testing (MBT) generates test cases through a model representing the software under test (SUT). The generated abstract test cases need to be transformed into concrete or executable test scripts. Despite the benefits offered by MBT, its industrial adoption is slow. This paper aims to propose a Model-Based Test scrIpt GenEration fRamework (TIGER) based on GraphWalker (GW), an open-source MBT tool, to evaluate the accuracy of generated test scripts to reflect real-world scenarios defined by the model, and to report on the findings of an industrial survey on MBT adoption. We have validated the robustness of the TIGER using an industrial case study from Alstom Rail AB, Sweden. We have injected faults into the model of the SUT based on three mutation operators to generate faulty test scripts. The aim of generating faulty test scripts is to produce failing test steps and to guarantee the absence of faults in the SUT. Moreover, we have also generated the test scripts using the correct version of the model and executed them to analyze the behavior of the generated test scripts in comparison with manually written test scripts. The experimental results show that the generated test scripts are executable, provide 100% requirements coverage, and can be used to uncover faults at the software-in-the-loop simulation level of system testing. Additionally, the analysis of the survey data reveals that MBT can address most of the identified testing challenges, but there remain certain barriers to its adoption.

Translating programmable logic controller (PLC) programs into analyzable software artifacts is an important step toward enabling modern software testing and verification techniques in industrial settings. This paper presents a replication study in which we use PyLC+, a translation framework for IEC 61131-3 Function Block Diagrams (FBD), together with the Nagini verifier to perform functional correctness checks on industrial safety-critical logic. PyLC+ extracts block networks from PLCopen XML and generates executable Python models that preserve block semantics, signal flow, and scan-cycle behavior. We applied this workflow to a representative set of 13 industrial POUs from an electropneumatic brake control subsystem. All 13 POUs were verified: twelve using concrete or partially abstracted FBD models, and one using an abstract requirements-level specification whose 7-tuple specification initially exposed a bug in the Nagini verifier (a lack of support for tuples with more than six elements), which was fixed by the Nagini developers. The results show that a combination of automated extraction, manual semantic reconstruction, and contract-based specification is sufficient to verify realistic industrial logic at the POU level. We also discuss the abstraction decisions, the Nagini bug uncovered by one POU, and the remaining missing block semantics that constrain how far automation can be pushed in future versions of PyLC+.

Regulatory standards for engineering safety-critical systems often demand both traceable requirements and specification-based testing, during development. Requirements are often written in natural language, yet for specification purposes, this may be supplemented by formal or semi-formal descriptions, to increase clarity. However, the choice of notation of the latter is often constrained by the designers’ training, skills, and preferences. The Easy Approach to Requirements Syntax (EARS) addresses the inherent imprecision of natural language requirements concerning potential ambiguity and lack of accuracy. This paper investigates requirements specification using EARS, and specification-based testing of embedded software written in the IEC 61131-3 language, a programming standard for developing programmable logic controllers (PLC). Further, we study, utilizing an experiment, how human participants translate natural language requirements into EARS and how they use the latter to test PLC software. We report our observations during the experiments, including the type of EARS patterns that participants use to structure natural language requirements and challenges during the specification phase, and present the results of testing based on EARS-formalized requirements in real-world industrial settings.

Automated test generation is an area that has seen a lot of research and development, resulting in many test automation methods and tools for test design. However, practitioners often face challenges in adopting these tools. This is not only due to the immaturity of some tools but also because of varying perspectives, confusing terminology, and, most importantly, the lack of a clear framework to guide the selection of the most suitable approach for their needs. We propose a taxonomy that characterizes the methods for automated test generation. The taxonomy was constructed using a process that involved analyzing secondary studies on automated test generation and existing taxonomies in the scientific literature. Direct observations and iterative refinements were included, followed by validation through conceptual evaluation and practitioner feedback. The resulting dimensions characterize automated test generation and its use in software testing. The taxonomy is organized into several dimensions: software artifact (i.e., type, notation, interfaces), test generation (i.e., objectives, methods, and monitoring), test execution, and test oracle. We demonstrate the taxonomy’s use by applying it to several automated test-generation tools. This paper provides the necessary concepts and a generic process for categorizing and assessing automated test generation approaches. © IFIP International Federation for Information Processing 2025.

Software quality assurance is critical in safety-critical domains like railway systems, where failures can have catastrophic consequences. In this context, train control and management systems play a central role, and their software must undergo rigorous validation. Alstom Rail Sweden AB employs a digital twin infrastructure to simulate and validate train control and management systems software. While this setup significantly improves system-level testing, the root cause analysis of test failures remains a manual and time-consuming bottleneck.This study explores the potential of large language models to automate root cause analysis by interpreting execution logs generated during digital twin-based testing. We benchmark seven state-of-the-art large language models, Aion-1.0, DeepSeek R1, DeepSeek V3 0324, Mistral Small 3.1 24B, GPT o3-mini, Gemini 2.5 Pro Experimental, and QwB 32B, using zero-shot chain-of-thought prompting to assess their ability to reason about fault patterns in real-world industrial test execution logs. The logs, sourced from Alstom’s digital twin-based testing environment, capture complex operational behaviour typical of embedded, safety-critical systems.Our results show that Gemini 2.5 Pro Experimental achieved the best performance with 66.7% accuracy and strong reasoning quality in this domain, contributing to the future research agenda to improve the accuracy prediction.

Text mining techniques, particularly those leveraging machine learning for natural language processing, have gained significant attention for qualitative data analysis in software testing. However, their complexity and lack of transparency can pose challenges, especially in safety-critical domains where simpler, interpretable solutions are often preferred unless accuracy is heavily compromised. This study investigates the trade-offs between complexity, effort, accuracy, and utility in text mining and clustering techniques, focusing on their application for detecting functional dependencies among manual integration test cases in safety-critical systems. Using empirical data from an industrial testing project at ALSTOM Sweden, we evaluate various string distance methods, NCD compressors, and machine learning approaches. The results highlight the impact of preprocessing techniques, such as tokenization, and intrinsic factors, such as text length, on algorithm performance. Findings demonstrate how text mining and clustering can be optimized for safety-critical contexts, offering actionable insights for researchers and practitioners aiming to balance simplicity and effectiveness in their testing workflows.

A cross-sectional, questionnaire-based survey of software testing courses offered at Swedish universities was undertaken in the final quarter of 2023. With a return rate of 44%, the survey delved into the contents of these software testing courses to gain an understanding of how the courses differ in terms of depth and breadth of content. Information was also sought about administrative and course planning activities related to the courses. Some key findings are that there is in-depth coverage of unit testing in all the courses, with none of the courses offering in-depth testing of other test levels such as acceptance testing. Also notable is the difference in test types. As an example, functional testing is taught in-depth in all the courses, while accessibility testing is not taught at all in half of the courses. It is suggested that a greater range of software testing topics is needed in future education if more stakeholders, such as business analysts and software developers, not just software testers, are to have a quality-centred approach to software development.

As industrial PLC programs become more complex, automated testing and verification methods are needed to ensure their reliability and correctness. This paper presents PyLC+, a modular framework that translates PLC programs into Python, allowing for automated AI-driven test generation. PyLC+ builds upon our previous work, addressing limitations by adopting a class-based modular architecture that improves the tool's scalability, maintainability, and extensibility. This structural refinement eliminates reliance on nested functions, facilitating the translation of large-scale, real-world PLC programs while maintaining precise use of cyclic execution. Furthermore, PyLC+ introduces automated handling of stateful FBs, ensuring compliance with IEC 61131-3 execution semantics.Additionally, the tool proposes integrating LLM-driven test generation with search-based test generation to improve the efficiency and effectiveness of testing PLC software. We tested PyLC+ in a large-scale company developing train control systems, demonstrating its efficiency and effectiveness in handling complex industrial PLC programs.

Phishing constitutes a form of social engineering that aims to deceive individuals through email communication. Extensive prior research has underscored phishing as one of the most commonly employed attack vectors for infiltrating organizational networks. A prevalent method involves misleading the target by employing phishing URLs concealed through hyperlink strategies. PhishTank, a website employing the concept of crowd-sourcing, aggregates phishing URLs and subsequently verifies their authenticity. In the course of this study, we leveraged a Python script to extract data from the PhishTank website, amassing a comprehensive dataset comprising over 190,0000 phishing URLs. This dataset is a valuable resource that can be harnessed by both researchers and practitioners for enhancing phish- ing filters, fortifying firewalls, security education, and refining training and testing models, among other applications.