Verification & validation & levels of software testing

Verification & validation & levels of software testing

Static Testing Dynamic Testing Testing done without executing the program Testing done by executing the program This testing does verification process Dynamic testing does validation process Static testing is about prevention of defects Dynamic testing is about finding and fixing the defects Static testing gives assessment of code and Dynamic testing gives bugs/bottlenecks in documentation the software system. Static testing involves checklist and process Dynamic testing involves test cases for to be followed execution This testing can be performed before compilation Dynamic testing is performed after compilation Static testing covers the structural and statement coverage testing Dynamic testing covers the executable file of the code Cost of finding defects and fixing is less Cost of finding and fixing defects is high Return on investment will be high as this process involved at early stage Return on investment will be low as this process involves after the development phase More reviews comments are highly recommended for good quality More defects are highly recommended for good quality. Requires loads of meetings Comparatively requires lesser meetings

Types of software testing (verification & validation) Verification tests are also known as static tests as no execution of artifact is required to run it during these tests. Verification tests include requirement specification reviews, design reviews, source code reviews etc. Validation tests are also known as dynamic tests as execution of artifact under test is required to run it during these tests. Validation tests are further divided into white box and black box tests. Verification and Validation are two measures used to check that the software product meets the requirements specifications. Together they help improve software quality.

Types of software testing (verification & validation) White box tests are named so as the source code is used during software testing. White box tests are also known as structural tests as they test small parts (structures) of the software product. Unit tests and integration tests fall under these tests. Black box tests are named so as the source code is not used during software testing. Instead the executable binary machine code is used. Black box tests are further divided into functional and non functional tests. Non functional tests are further divided into performance, security, usability etc. tests. Black box tests are done both at system level as well as at deployment (user acceptance) level. Regression tests are done for both white box as well as black box tests.

Verification & validation and associated software testing types

Verification & Validation Verification is that part of software testing where the testing is done on any software project artifact which has no executable part available. Software requirement specifications, software designs, source code (testing to view only the source code and not running it) are project artifacts which are tested using verification techniques. Requirement specifications are tested using requirement specification reviews, software designs are tested using design reviews and source code (construction) is tested using source code reviews.

Verification & Validation Validation is that part of software testing where the testing is done on software product by executing its source code. Business logic at class (component) level is tested using unit testing, integration of component is tested using integration testing, the software product is tested using system testing and finally the deployment of the software product is tested by end users using user acceptance testing.

Levels of software testing (validation)

Chapter 9 Software Testing (Verification & Validation) Software testing (validation) is done at many levels. These levels include unit, integration, system and deployment. All of these levels come under validation. Software components are tested for business logic correctness at unit tests level. These components are then tested for integration with source code build using integration testing. The completely built software product after integration of all components is again tested at system level. At this level functional tests are done to test if software product features are working as per software requirement specifications. When the software product is ready then it is deployed at client site. The end users do user acceptance testing to ensure that the software product is working as per requirement specifications.

Chapter 9 Software Testing (Verification & Validation) Barry Boehm’s V Model for software testing

Software Testing Barry Boehm’s V Model is all about linking artifacts developed during software development lifecycle with levels of testing. Software requirement specifications are used to perform user acceptance testing. Software design specifications are used to perform system testing. Software unit construction (software component) is used to perform unit testing. Software module or sub system is used to perform integration testing.

Static testing: Manual or automated reviews of the documents . Also known as Non-execution technique. This testing is done during initial phase of testing to catch Defects early in SDLC. It examines work documents and provides review comments Work document can be of following: Requirement specifications Design document Source Code Test Plans Test Cases Test Scripts Help or User document Web Page content

Software verification Verification tests include requirement specification reviews, software design reviews, source code reviews etc. Requirement specification reviews are done after these artifacts get completed. Software testers or business analysts review the requirement specifications for completeness, ambiguity and errors. If any errors or incompleteness is found then it is rectified. Software design reviews are done to check if the software design is correct as per requirement specifications. Software design reviews also include reviewing software component structure. If large classes were with many methods are designed then it will lead to difficult to understand source code. Similar structural problems can be found during design reviews. The identified design problems will need to be rectified . Source code reviews are done to check for dead code, wrong implementation of business logic etc. If any of these defects are found then they are rectified. Code walkthroughs are also used for source code checking. Here the developer presents his/her source code in front of other project team members. Team members review the source code and find any defects are there in the code. Software developer later rectifies those defects.

Unit testing Unit testing is done after a class or component is constructed. Each class implements some business logic. The source code in the class is checked to verify if the business logic is implemented properly. Unit testing is also done for extended testing when database access is also involved. In such cases business logic as well as database access aspect also need to be considered. Testing database connection and testing if database operation actually happens are also done on these tests.

Example package testTry; public class Box { int x; int y; public Box(int xx, int yy) { x xx; y yy; } int boxarea() { return x*y; } public static void main(String[] args) { // TODO Auto-generated method stub Box b new Box(4,5); System.out.println("Area of my Box is " b.boxarea()); } }

Example Test class package testTry; import static org.junit.jupiter.api.Assertions.*; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; class boxTest { Box b; @BeforeEach void setUp() throws Exception { b new Box(3,4); } @Test void test() { assertEquals(12, b.boxarea()); } }

Unit Testing Example class addition { public boolean adding (integer one, integer two){ if one 1 and one 100 and two 1 and two 100 { integer x one two; print (x); return true; } else return false; } } The above class implements a business logic which needs to be tested. The test code can be as following: class testAddition { public method void testAdding () { boolean x addition.adding(-3, 5); } } The above test case is a negative test case as the expected result should fail. Using positive and negative tests, we can find out if the business logic works fine. For positive and negative testing, this test class can be provided with various values and these values will be passed to the class under test through parameters.

Unit Test: Example public class interest compute { public integer interest compute (float interest) { integer principal 293; integer accrued amount; accrued amount principal principal x interest/100; return accrued amount; } } In some cases, the business logic seems to be fine but its implementation is incorrect. The above example will compute the result correctly but the final result is incorrect as due to faulty variable declaration, the final computation is truncated.

Software Testing (Verification & Validation) class data access { public delete row () { string y ‘Ammar’; integer x; search in the database table and find the record number where customer name is Ammar and assign it to x; if x record number of Ammar then delete row from customer table where customer name y; } } When a class involves database access then apart from unit testing, database connection needs to be checked as well as to find out if database operation also took place. In the above example, you will also need to find out if the mentioned record has been deleted from the database.

Database operations work on the principle of first searching a particular record and then doing database operations. If a particular record is changed or deleted after a test then the same test script will not work next time. Since the particular database record was changed after execution of test script the first time, next time you may need to change test script. If database connection and database operations are not needed to be tested then the database environment can be simulated using tools.

Integration testing Integration testing is used to find out if a newly created class or component or module can be integrated correctly with the rest of the software product. Integration testing thus is a test of the interfaces of a class or component or module. If the interface of the class or component or module is correct then the class or component or module will integrate with the rest of the software product.

Integration testing in continuous integration of source code build environment

Continuous integration Continuous integration of source code build is used on agile projects. Here when a unit of source code (class or component) is written by developers is integrated with the source code build. Integration testing is performed before a piece of source code is integrated with the source code build. If integration testing fails then the unit piece of source code is checked if it has any interface problems due to which it is not able to integrate. Corrected piece of source code is again tested for integration. This continues till there are no integration failures. Once integration test passes then this piece of source code is integrated with the source code build.

Frequency of integration of source code

Explanation On Waterfall projects, separate modules of software product are developed first. They are later integrated with each other. On agile projects however, source code is continuously integrated. These 2 strategies imply that frequency of integration of source code vary considerably. Whereas in case of Waterfall projects, frequency of integration of source code is infrequent; in case of agile projects, the frequency is very high. In fact on agile projects, frequency of integration of source code with the source code build is 100%. Every time a new class is created, it is immediately integrated with the source code build.

Explanation This discussion implies that integration testing on agile projects is done frequently. This leads to a better software product with less number of integration software defects. When modules of software product are tried to be integrated with each other then a large number of integration problems will arise. In this scenario, it is also difficult to pin point the cause of integration problems. Thus fixing integration problems is also difficult and it takes time in fixing them. This is the reason, software projects are increasingly using continuous integration of source code.

System, regression and user acceptance testing Many types of testing are done on software projects during system and user acceptance testing. Functional testing is done to test if software product features are working as per software design and requirement specifications. Non functional testing is done to test if software product is working under acceptable limits even when environmental conditions are deteriorated. If a website receives tremendous web traffic but even with so much traffic, the website is able to respond user page requests within acceptable limit then performance of this website is considered good. Similarly if a website receives malware attacks but is still able to overcome these attacks well and keeps running then security of this website is considered good. Non functional tests include performance, security, usability, localization etc. types of tests. Regression tests are also performed during system testing for software products which are built incrementally. Regression tests check if existing functionality of a software product features still work after integration of increment of software product features.

Software testing done in software development, deployment and maintenance

Software testing is not only done during software development but also during deployment and maintenance. During software development all verification and validation tests are performed (requirement specification reviews, design review, code reviews, unit, integration, system (functional and non functional tests). Regression tests are also performed. But during deployment, only user acceptance testing is performed. During deployment, all black box testing (functional and non functional) tests are performed. The goal here is to validate that the software product works as per the requirement specifications. After deployment a software product is used by software users. This is the production phase of the software product. During production, software product is tested for performance and sanity tests. When the software product is taken for maintenance then regression tests are also done.

Alpha-Beta testing cycle

Other important software tests If alpha and beta release of a software product are done then alpha and beta testing is done for the software product. Alpha testing is a form of internal acceptance testing performed mainly by the in-house software QA and testing teams. Alpha testing is the last testing done by the test teams at the development site after the acceptance testing and before releasing the software for beta test. Alpha testing can also be done by the potential users or customers of the application. But still, this is a form of in-house acceptance testing.

Beta testing. This is a testing stage followed by the internal full alpha test cycle. This is the final testing phase where the companies release the software to few external user groups outside the company test teams or employees. This initial software version is known as the beta version. Most companies gather user feedback in this release. In short, beta testing can be defined as– the testing carried out by real users in a real environment.

SMOKE TESTING, also known as “Build Verification Testing”, is done when a piece of source code is integrated with the source code build. It is a type of software testing that comprises of a nonexhaustive set of tests that aim at ensuring that the most important functions work. The result of this testing is used to decide if a build is stable enough to proceed with further testing. Advantages It exposes integration issues. It uncovers problems early. It provides some level of confidence that changes to the software have not adversely affected major areas of code. Levels Smoke testing is normally used in Integration Testing, System Testing and Acceptance Testing levels.

Planned testing involves writing test cases and then executing them to find software defects. Planned testing has a limitation that only limited testing can be done within the allowed time limits for testing.

Exploratory testing removes this barrier. Instead of spending too much time in writing test cases, exploratory testing does not use test cases at all. A software tester will just run various commands to execute business processes and find out if they are working correctly. If the tester finds any incorrect behavior then he/she will file a test defect with the details. Using this technique more testing of the software product can be done within a short span of time.

coverage based testing A coverage based testing strategy is used when it is known in advance as to which part of the software product needs to be tested. This strategy is adopted when it is known that it is not possible to test the entire software product in the allotted time for software testing on a software project. A good strategy in such situation is to prioritize test cases and do testing only for test cases which are high on priority.

Penetration testing is a usually a form of black box security testing. In a pen test you are trying to break into the server as many times as possible and report back on how they where able to break in. This is often done multiple times to make sure that the patches hold water. This is an important type of security testing because its as real world as it gets. (Common tools for web application penetration testing are Acunetix)

Chapter 9 Software Testing (Verification & Validation) Test case description A test case consists of test case description, entry criteria, exit criteria, expected result, actual result and test execution result (pass/fail). Test case description contains steps which are needed to be done during execution of a test case. Entry criteria is the conditions which should be valid before a test case can be executed. Exit criteria is the conditions which should be valid after a test case is executed. Expected result is the expected outcome of computation after execution of a test case. Actual result is the actual outcome of computation after execution of a test case. If actual result is same as expected result then the test case passes and it is assumed that the tested business process has worked as per expected. Thus there is no software defect. In contrast if the actual result is not the same as expected result then it is assumed that the tested business process has not worked as per expected. Thus there is a software defect and it needs to be fixed.

Chapter 9 Software Testing (Verification & Validation) Test case design Designing good test cases is important because all testing needs to be carried out within time limits. If test designs are not good then it may lead to ineffective testing. Even with limited number of good test cases it is possible to test a software product effectively. Using boundary value analysis it is possible to test a business process with just 3 test cases. In one test case, a test value which is above the allowed range of valid input can be used for a negative result and assess if the business logic works correctly. Similarly a value below the allowed range of valid input can be used for a negative result and assess if the business logic works correctly. Finally a value within the allowed range of valid input can be used for a positive result and assess if the business logic works correctly. Using decision trees it is possible to test complex business logic where a matrix of values are needed to test a business process.

Chapter 9 Software Testing (Verification & Validation) Test environment A good test environment is important for finding software defects. If a good test environment is not provided then it can lead to difficulty in finding defects. Once a software release needs to be tested then a stable and isolated test environment is created on a computer. Exact replica of the software product is installed on this environment. Test data is also created. Apart from the test environment, supporting facilities like defect tracking system, configuration management system etc. also need to be prepared for a testing cycle.

Chapter 9 Software Testing (Verification & Validation) Test cycle

Chapter 9 Software Testing (Verification & Validation) Test cycle is the process which is followed from test planning to test case writing to test execution to defect logging. Test planning involves finding out which software features need to be tested, how testing will be done and for how long testing activities will be carried out. Test case design involves finding out what testing techniques (boundary value analysis, decision trees etc.) will be used and how much test coverage will be done during test cycle. Test cases are written by testers after test design is decided. How many test cases needs to be written will depend on test coverage, testing techniques used etc. Test case execution starts when test preparation is done and test cases have been written. All the failed test cases will result in pointing to software defects. These software defects are logged so that they can be fixed.

Chapter 9 Software Testing (Verification & Validation) Software defect cycle

Chapter 9 Software Testing (Verification & Validation) When test case execution starts then some of the test cases will pass while others will fail. The failed test case point to software defects. These software defects are logged in a defect tracking system. The software developer who owns the software product feature which contains the software defect is notified. The software developer fixes the software defect and notifies the software tester who had logged the software defect. The tester then verifies if the software defect indeed has been fixed. If the defect is found to be fixed then the tester will close the software defect in the defect tracking system. This is the complete lifecycle of software defect and its fixing.