Why Software Testing Fails: Common Pitfalls Observed in a Critical Smart Metering Project

Over the last decades a considerable share of software engineering research has been dedicated to the area of software testing. Still, however, testing often fails or causes major problems in practice. In this paper we present insights and experiences from a large project in the energy sector. The obligatory switch from analog energy meters to smart metering technology poses a big challenge for many energy providers. Apart from technical issues concerning meters and transmission technology, the adaption of the internal business processes together with the development of backend software can turn out to be more difficult than expected. The criticality, size and complexity of the analyzed project are reflected in software and system testing, where the underestimated effort, mistakes, and wrong decisions caused serious difficulties. In our work we describe the observed testing problems and the underlying causes. Subsequently, we compare the identified problems with a catalogue of commonly known testing pitfalls and anti-patterns. The results show that the majority of the observed problems are not new or specific to the studied project. Furthermore, additional candidates for extending the list of common pitfalls are identified. Besides recommendations on how to mitigate the problems in the studied project, we conclude with the general insight that there is a great potential to improve software testing practice by developing measures for early recognition, communication, and avoiding of common mistakes.

Authors: Stefan Mohacsi, Rudolf Ramler

Vorgetragen von: Stefan Mohacsi, Rudolf Ramler
Unternehmen: Atos IT Solutions and Services

Vortragssprache: Englisch
Level: Fortgeschrittene

Partner 2019

ASQF e.V ATB - Austrian Testing Board CON•ECT Eventmanagement GmbH dpunkt.verlag GmbH Heise Medien GmbH & Co. KG IREB GmbH iSQI GmbH IT Verlag GmbH IT-Cluster der Business Upper Austria GmbH Österreichische Computer Gesellschaft (OCG) TU Wien, Inst f. Information Systems Eng., CDL-SQ Verband Österreichischer Software Industrie (VÖSI)