Dependent input sampling strategies: using metaheuristics for generating parameterised random sampling regimes

TY - GEN

T1 - Dependent input sampling strategies

T2 - 2018 Genetic and Evolutionary Computation Conference, GECCO 2018

AU - Srivisut, Komsan

AU - Clark, John A.

AU - Paige, Richard F.

N1 - ©2018 Association for Computing Machinery. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details

PY - 2018/7/2

Y1 - 2018/7/2

N2 - Understanding extreme execution times is of great importance in gaining assurance in real-time embedded systems. The standard benchmark for dynamic testing'uniform randomised testing'is inadequate for reaching extreme execution times in these systems. Metaheuristics have been shown to be an effective means of directly searching for inputs with such behaviours but the increasing complexity of modern systems is now posing challenges to the effectiveness of this approach. The research reported in this paper investigates the use of metaheuristic search to discover biased random sampling regimes. Rather than search for test inputs, we search for distributions of test inputs that are then sampled. The search proceeds to discover and exploit relationships between test input variables, leading to sampling regimes where the distribution of a sampled parameter depends on the values of previously sampled input parameters. Our results show that test vectors indirectly generated from our dependent approach produce significantly more extreme (longer) execution times than those generated by direct metaheuristic searches.

AB - Understanding extreme execution times is of great importance in gaining assurance in real-time embedded systems. The standard benchmark for dynamic testing'uniform randomised testing'is inadequate for reaching extreme execution times in these systems. Metaheuristics have been shown to be an effective means of directly searching for inputs with such behaviours but the increasing complexity of modern systems is now posing challenges to the effectiveness of this approach. The research reported in this paper investigates the use of metaheuristic search to discover biased random sampling regimes. Rather than search for test inputs, we search for distributions of test inputs that are then sampled. The search proceeds to discover and exploit relationships between test input variables, leading to sampling regimes where the distribution of a sampled parameter depends on the values of previously sampled input parameters. Our results show that test vectors indirectly generated from our dependent approach produce significantly more extreme (longer) execution times than those generated by direct metaheuristic searches.

KW - Genetic algorithms

KW - Hill climbing

KW - Metaheuristics

KW - Simulated annealing

KW - Temporal testing

UR - http://www.scopus.com/inward/record.url?scp=85050641560&partnerID=8YFLogxK

U2 - 10.1145/3205455.3205495

DO - 10.1145/3205455.3205495

M3 - Conference contribution

AN - SCOPUS:85050641560

SP - 1451

EP - 1458

BT - GECCO 2018 - Proceedings of the 2018 Genetic and Evolutionary Computation Conference

PB - ACM

Y2 - 15 July 2018 through 19 July 2018

ER -