Practice 5: Verify Software Quality Develop Iteratively

Practice 5: Verify Software Quality Develop Iteratively Manage Requirements Use Component Architectures Model Visually Verify Quality Control Changes Quality, as used within Rational Unified Process, is defined as “The characteristic of having demonstrated the achievement of producing a product which meets or exceeds agreed upon requirements as measured by agreed upon measures and criteria And is produced by an agreed upon process. If done this way, the process can be repeated and managed In most organizations, testing accounts for 30-50% of development costs! Yet most people believe software is not adequately tested when delivered. Testing is difficult; complete testing is impossible; a good process and automated tools help! Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 1

Practice 5: Verify Software Quality Software problems are 100 to 1000 times more costly to find and repair after deployment Cost Development Deployment Test early and continuously! Test functionality, reliability; performance; Test architectural decisions early. Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 2

Iterative Development Permits Continuous Testing Iteration 1 Iteration 2 R R D R D D C T I M E Test Life Cycle Iteration 3 C C T T T Test Test Test Plan Design Implement Execute Evaluate Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved Plan Design Implement Execute Evaluate 3 Plan Design Implement Execute Evaluate

Iterative Development – and Continuous Testing (cont.) Rather than test one time, spread testing out continuously. Part of each iteration – BUT (see below) Each iteration produces an executable release (not a product release ) Don’t think of an ‘executable’ as just an .exe or .dll. The executables may be part of an architecture . Each iteration is tested and integrated into an evolving application. Note: Each ‘phase’ has iteration(s), and each phase has milestones! (A phase may have zero or more iterations prior to arriving at the phase milestone!) Be careful: The ‘degree’ of R, D, C, T depends on which phase the iteration is in! See your drawings of the RUP (There are MANY). See pg. 66 in RUP, third edition, as one of many examples ) Keep this handy! Will be referencing this many times. Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 4

More recall – talking about Verifying Quality Cannot ‘engineer in’ quality; Must be threaded throughout development! Notice: Continuous Testing and integration! Distributes testing . At end, entire system tested as a whole. Many errors can be found early and fixed while repair costs are inexpensive Architectural decisions (key decisions) tested early avoiding disastrous problems later. These features greatly reduce risks and liability of delivering poor quality systems. Early iterations are used to mitigate risk and address core functionalities (whether in elaboration, construction ) Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 5

Automated Tests Requirements Testing in an Iterative Environment Iteration 1 Iteration 2 Iteration 3 Iteration 4 Continuous integration!!! (one of the major problems of SDLC!) We will produce automated tests. As new requirements are added in iterations, new tests are generated and run. This means that some tests will be rerun – part of ‘Regression Testing.’ Test Suite 1 Test Suite 2 Test Suite 3 Test Suite 4 Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 6

Automation Reduces Testing Time and Effort One Manual Test Cycle 13,000 Tests 2 Weeks Test Automation 13,000 13,000Tests Tests 66hours hours 11Person Person Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 6 People Manual preparation of tests is very expensive and usually results in missed ‘opportunities.’ Run More Tests More Often 7

Dimensions of Software Quality Type Why? How? Functionality Does my app do what’s required? Test cases for each scenario implemented Reliability Does my app leak memory? Analysis tools and code instrumentation Application Performance Does my app respond acceptably? Check performance for each use-case/scenario implemented System Performance Does my system perform under production load? Test performance of all usecases under authentic and worst-case load For each iteration, do the ‘above’ software quality checks. Remember: tests are ‘driven’ by Use Cases and Supplementary Specifications! Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 8

Problems Addressed by Verifying Quality Root Causes Solutions Objective assessment exposes inconsistencies early (continuous integration helps!) Insufficient requirements Ambiguous communications Brittle architectures Overwhelming complexity Subjective assessment Undetected inconsistencies Poor testing Waterfall development Uncontrolled change Insufficient automation Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved Testing provides objective project status assessment Testing and verification are focused on high risk areas Defects are found earlier and are less expensive to fix (because ‘testing’ is distributed Automated testing tools provide testing for reliability, functionality, and performance 9

Practice 6: Control Changes to Software Develop Iteratively Manage Requirements Use Component Architectures Model Visually Verify Quality Control Changes Must recognize that we CANNOT STOP CHANGE The only ‘constant’ is ‘change!’ But, we must be able to Manage Change! Must control How and When control is introduced and who introduces the changes. This DOES NOT MEAN that we absolutely accept ALL changes, But (Discuss!) Want a process that can respond to change (RUP) Must synchronize Change across development teams and locations too. (What impacts do proposed changes have on our architecture!) Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 10

Practice 6: Control Changes to Software Multiple developers Multiple teams Multiple sites Multiple iterations Multiple releases Multiple projects Multiple platforms May have multiple developers organized into different teams at multiple sites all working together on multiple iterations, releases, products, and platforms (mostly based on the software architecture) Without explicit control, parallel development degrades to chaos!!!! Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 11

Change Control Supports All Other Best Practices Develop iteratively Manage requirements Use component architectures Model visually Progress is incremental only if changes to artifacts are controlled To avoid scope creep, assess the impact of all proposed changes before approval Components must be reliable, i.e., the correct versions of all constituent parts found To assure convergence, incrementally control models as designs stabilize Tests are only meaningful if the versions of the items under test are known and the items protected from changes Verify quality Italicized items – verbally discussed in class. Not necessarily more important than others Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 14

Problems Addressed by Controlling Changes Root Causes Insufficient requirements Ambiguous communications Brittle architectures Overwhelming complexity Subjective assessment Undetected inconsistencies Poor testing Waterfall development Uncontrolled change Insufficient automation Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved Solutions Requirements change workflow is defined and repeatable Change requests facilitate clear communications Isolated workspaces reduce interference from parallel work Change rate statistics are good metrics for objectively assessing project status Workspaces contain all artifacts, facilitating consistency Change propagation is controlled Changes maintained in a robust, customizable system 15

Best Practices Reinforce Each Other Ensures users involved as requirements evolve Manage Requirements Use Validates architectural Component decisions early on. Drives development, Architectures planning, change control. . Develop Iteratively Addresses complexity of design/implementation incrementally Need tools/support environment! Measures quality early and often Continuous testing and integration Evolves baselines incrementally Architecture teams localizing changes; Need CMS, Conf Control Model Visually Verify Quality Control Changes Remember: these best practices yield the best results WHEN USED COLLECTIVELY! Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved 16

Summary: Best Practices of Software Engineering The result is software that is On Time On Budget Meets Users Needs Performance Engineer Analyst Project Manager Develop Iteratively Developer Use Manage Component Requirements Architectures Model Visually Verify Quality Tester Control Changes Unified Software Practices v 5.0 - D Copyright 1998 Rational Software, all rights reserved Release Engineer 17