The York Research Database

Systematic software testing provides an important source of software failure analysis. The field suffers, however, from insufficiently reproducible results, lack of standard credible data, and insufficiently explicit assumptions. The present article attempts to provide an objective basis for failure analysis through an automatic testing framework (AutoTest) for contract-equipped software. We present five principles for scientific failure analysis, a set of reproducible test results, and a first analysis of their consequences for software development. 1 Testing strategies and their purpose The area of software testing provides one of the best possible illustrations of the lack of credible large-scale failure analysis highlighted by the call for proposals for this workshop.