Coverage Metrics for Saturation-based and Search-based Testing of Concurrent Software

• http://www.springerlink.com/content/084w5489p9556775/

Java, concurrency, testing, coverage, search-based testing, saturation-based testing

Coverage metrics play a crucial role in testing.They allow one to estimate how well a~program has been tested and/or to controlthe testing process. Several concurrency-related coverage metrics have beenproposed, but most of them do not reflect concurrent behaviour accurately enough.In this paper, we propose several new metrics that are suitable primarily forsaturation-based or search-based testing of concurrent software. Theirdistinguishing feature is that they are derived from various dynamic analysesdesigned for detecting synchronisation errors in concurrent software. In fact,the way these metrics are obtained is generic, and further metrics can beobtained in a similar way from other analyses. The underlying motivation isthat, within such analyses, behavioural aspects crucial for occurrence ofvarious bugs are identified, and hence it makes sense to track how well theoccurrence of such phenomena is covered by testing. Next, coverage tasks of theproposed as well as some existing metrics are combined with an abstractidentification of the threads participating in generation of the phenomenacaptured in the concerned tasks. This way, further, more precise metrics areobtained. Finally, an empirical evaluation of the proposed metrics, whichconfirms that several of them are indeed more suitable for saturation-based andsearch-based testing than the previously known metrics, ispresented.