RETORCH: Resource-Aware End-to-End Test Orchestration

Abstract

Continuous integration practices introduce incremental changes in the code to both improve the quality and add new functionality. These changes can introduce faults that can be timely detected through continuous testing by automating the test cases and re-executing them at each code change. However, re-executing all test cases at each change may not be always feasible, especially for those test cases that make heavy use of resources thoroughly like End-to-End test cases that need a complex test infrastructure. This paper is focused on optimizing the usage of the resources employed during End-to-End testing (e.g., storage, memory, web servers or tables of a database, among others) through a resource-aware test orchestration technique in the context of continuous integration in the cloud. In order to optimize both the cost/usage of resources and the execution time, the approach proposes to (i) identify the resources required by the End-to-End test cases, (ii) group together those tests that need the same resources, (iii) deploy the tests in both dependency isolated and elastic environments, and (iv) schedule their parallel execution in several machines