An Agile Testing Framework of Four Quadrants

The research is based on the idea of agile testing, taking the “order cancelled” module of an e-commerce website as an application case, this study is devoted to exploring a feasible and scientific agile testing framework of four quadrants to form an anticlockwise closed loop for guiding the software agile testing process. Furthermore, further work is put forward to establish of a special defect management database for bug definition and analysis, and build the particular test process models for each quadrant of this agile testing framework gradually.


Introduction
Agile testing is a rapid testing practice that follows the agile manifesto, emphasizing testing systems from the user's perspective, focusing on the newly developed capabilities of iterative testing instead of emphasizing the rigorous testing phase of traditional testing [1][2]. Agile testing is inherited from quite a few agile development models obviously. Testing is priority, the speed and adaptability in testing are poured attention into the goal on user requirement, and the final software product could be released in time [1]. The development of agile testing has a history of nearly seventeen years, and Figure 1 demonstrates an agile test matrix initially. Agile test matrix can be considered as the prototype of agile testing framework, the four quadrants show that testers can work hard in these four areas. It does not reflect the positive values of the agile, and the principles of agile testing and even the way of thinking of agile testing are not reflected from this agile test matrix [3]. Scrum is a typical agile framework with iterative incremental software development process for completing complex projects [4], and Figure 2 demonstrates the scrum model for agile development. Agile testing has been widely used in various test practice, Driven Development (TDD), Acceptance Test Driven Development (ATDD), and etc., and a series of testing models which are based on agile are also come into being [5]. Figure 3 demonstrates an agile testing process, and Figure  4 demonstrates a common Scrum model for agile testing. New feature testing of continuous iteration are focused on agile testing, and sufficient acceptance testing is executed for the new features, while the regress testing of the original features is depended on automated testing. Except for acceptance testing, characteristics of agile testing in other parts of the scrum model in Figure 4 are not embodied positively [6]. Although, Scrum model for agile testing is widely used for testing and management process of agile projects in many references, lack of guidance on flow work of business and testing techniques.  This paper devotes to designing of a new agile testing framework of four quadrants. The structure of the paper is organized as the following: Section 2 describes the elements of test flows on each quadrant of this agile testing framework. An application case of this new agile testing framework of four quadrants is given in Section 3. Section 4 concludes the paper and puts forward the further work.

Agile testing framework of four quadrants
How to drive the team to build a high-quality software product will be more valuable in agile testing. Figure 5 demonstrates a new agile testing framework of four quadrants, which is quoted from [3], and it builds and evaluates product quality based on business layer and technical layer. Testing framework is driven by the business, the business must be the first, that user feedback is collected and analysed. Finally, user feedback is input into the user requirement to form an anticlockwise closed loop, which is more scientific and reasonable for agile testing [3].

Build Product Quality Based on Business Layer
In quadrant 1, business-driven testing includes Acceptance Test Driven Development (ATDD) and Behavior Driven Development (BDD), use case requirement, test driven design, etc. It not only clarifies and verifies the product requirement and design, but also completes building high-quality requirement and design of the product. This is more crucial and valuable.

Build Product Quality Based on Technology Layer
In quadrant 2, building product quality based on the technical layer, it focus on Continuous Integration (CI) technology and environmental support, realize unit test driven development, good automated unit testing, code static analysis, CI-based code review process, CI and Build Verification Test (BVT) of automatic and continuous, etc. to build high-quality code of the product.

Evaluate Product Quality Based on Technology Layer
In quadrant 3, product quality is evaluated based on the technical level, such as tool-based modeling, evaluation, monitoring and analysis of "functionality, performance, security, reliability", etc. It also relies on technology and testing infrastructure of DevOps. Quadrant 3 also fully reflects the testing technicality.

Evaluate Product Quality Based on Business Layer
In quadrant 4, product quality is evaluated based on business level, includes software exploratory testing, public testing, sprint review, etc. For the relevant stakeholders of the product, it is indispensable to evaluate the quality of the software product from the perspective of business, and these practices also conform to the principles of agile testing.

Application
In this section, the agile testing framework of four quadrants is applied in agile testing of an ecommerce website. As the limitation of length of paper, "order cancelled" module in this agile testing framework is introduced only. Table 1 demonstrates agile testing activities of this module in one test Sprint.  Table 2 demonstrates the user story division based on the requirement of "order cancelled" module, test case could be designed according to the user story. Automated testing environment of "Selenium" is configured to develop automated test scripts. Figure 6 demonstrates automated testing environment of Selenium. An order is cancelled by background system because of abnormal quantity. H An order is cancelled by background system because of abnormal inventory. L An order is cancelled by background system because of abnormal distribution. M "Keyword Driven" is used in the automated testing framework of this agile testing, the external excel file will be called when the test case is executed, and Figure 7 demonstrates a template of external excel file [7]. Because of agile development of this e-commerce website, some new functions will be added to the module in each Spring. The functionality points for coding needs to leave some buffers for the refactor part in process of unit test and build verification test. For the website, the tool of LoadRunner is used for performance test by recording script in Visual User Generator. For the configuration process, it won't be introduced in this paper. Finally, test experience can be combined with similar website applications, exploratory testing is executed for this module.  [8]. Exploratory testing methods could be chosen by testers.  Table 3 is quoted from [8], and the test value, content and steps for each method of exploratory testing are described in [9][10].)