From Diego Lo Guidice's Continuous Delivery Conference keynote:
"The term everyone is using is DevOps, but I think it should really be DevTestOps. Testing is really a crucial for achieving quality @ speed.
Forrester's 2015 State of Agile Development study revealed the following about leading firms' testing organization structure:
- 61% - Testers are allocated full-time to project teams
- 57% - Testers are members of an integrated delivery team
- 41% - Performance and load testers are centralized
- 36% - Have a Testing Practice Center of Excellence with specialized skills
- 35% - Performance and load testers are in project teams
- 29% - Have a Testing Center of Excellence where testing is managed and executed
- 28% - Automation architects are shared across projects
- 18% - Testing is managed and executed in the Testing Center of Excellence
Clearly, the leading organizations are making testers part of an integrated team. They don't have a test center of excellence decoupled from development. Testers are truly part of the team.
If you're doing Agile, you need to be able to complete all the testing in your iteration, and your "definition of done" needs to cover testing. Ideally, within each iteration, you're doing unit testing, functional testing, and even performance testing—this is all shifting to the left. You also need to ensure that you're using the right test data. You can have a great testing process and great development process, but if you’re not using the right test data, quite often the quality will be poor.
It's also interesting to look at the testing practices in use in leading organizations who are practicing agile:
- 82% - Manual testing
- 78% - Unit testing
- 67% - User acceptance testing in sprints
- 66% - Functional regression test automation
- 54% - Functional test automation
- 43% - Exploratory testing
- 42% - Test data management – Production data sub-settings, etc.
- 32% - Test data management – Masking, Obfuscation
- 25% - Test data management – Synthetic Generation
- 24% - Test-Driven Development (TDD)
- 19% - Service virtualization
- 14% - Behavior-Driven Development (BDD)
More Continuous Delivery Keynote Excerpts
Want to learn more from Diego Lo Guidice's Continuous Delivery Conference keynote? Check out these earlier blogs and videos:
- Forrester: Why Continuous Testing is Vital for Quality @ Speed: How testing needs to evolve in order to enable quality at speed for Agile and DevOps.
- Forrester: Barriers and Best Practices for Scaling Agile: The top 3 reasons Agile fails, how "Agile Expert" firms overcome common barriers, and Agile best practices.