Cognitive Heuristics in Software Engineering: Evaluating Confirmation Bias during Software Testing

Abstract

Software engineering is a human centric activity and the thought processes of software engineers have influence on the quality of software products. Cognitive scientists have identified human errors known as cognitive heuristics which could impact quality of a software product. However, there is little empirical evidence to substantiate this assertion in software engineering. In this research we study a specific heuristic and evaluate its impact in software testing. One of the factors that lead to poor quality of testing is to only verify if the system works as expected and ignore negative tests. This can be attributed to a heuristic called as confirmation bias which is defined as the tendency of people to verify their hypothesis rather than refuting them. The experiment design evaluates confirmation bias of software testers and measures their quality of testing. The results indicate that testers with low confirmation bias obtain better overall testing results.