Modern software testing practices deserve modern analytics, too. Learn how to go beyond your ordinary reports-and-dashboards tooling for software testing.
It wasn’t until I started working on Parasoft’s new website that I discovered the value of the behemoth that is Parasoft DTP*, and realized that we’ve been hiding this incredibly powerful technology from you for a while now. And I decided — it’s time for DTP to come out of the closet.
*Did you want to know what DTP stands for? Well... it doesn’t really stand for anything. The truth is, it used to be something else. When the product was named, it was an acronym for the Development Testing Platform. But since the product has evolved, we scrapped the full name and just use its initials now. Sorry to disappoint.
So, if it’s not a development testing platform, what is it?
Well, in fact, it’s a reporting and analytics tool that is so exciting, I decided to write this blog post.
What’s so exciting about DTP?
Let’s start at the basics. Parasoft DTP is way more powerful than your ordinary reports-and-dashboards tooling that you might expect from your static analysis tool or from other testing analytics products. DTP represents a real shift in what you can expect to get out of reporting and analytics for software testing.
Why? Because of the granularity of the data it captures, and the fact that it can capture data from all testing techniques (not just static analysis, but also data from manual testing, automated functional testing, etc.) and all tools that you’re using (not just the tools in the Parasoft ecosystem, but other tools you’re using, too). It’s also super flexible and customizable, so you can essentially measure anything your heart desires, for a comprehensive understanding of the health of your codebase, at a moment in time, or over time, across builds. And most importantly, because of all of this, it helps you quickly understand where to focus your limited resources to increase quality.
That is, after all, the whole point.
Beyond Simple Reports & Dashboards
So what does it mean to go “beyond” simple reports and dashboards? I would argue that there are 5 main ways that DTP moves you beyond, into something much more powerful that you can leverage across your SDLC.
I will enumerate them below.
1 — The Data DTP Collects
DTP captures an enormous amount of data across all of your testing practices – it’s truly unprecedented. Not just static analysis and code metrics, but also unit testing, functional testing, manual testing, code coverage, and code reviews. It aggregates all of this data together in order to give you a comprehensive view of the results of your testing practices.
In addition to aggregating all of this data, the data itself is collected in a granular way, enabling advanced computations and insights; flexibility and endless customization possibilities; and the ability to store a large amount of information over time. For example, because of the detailed data that DTP keeps on test execution, it can show you which tests executed against a specific file to achieve code coverage.
2 — How You Can Navigate This Data
After collecting your testing data, DTP gives you an interactive framework for navigating and analyzing these results. You might start in a somewhat standard report, but each report in DTP is made up of little customizable widgets, and when you click into one of them, you arrive at what is called an “Explorer” in DTP — an interactive UI for exploring your test data and understanding how changes in your codebase are changing code quality.
There are several Explorers in DTP, including the following examples:
- The Change Explorer lets you compare two builds and compare what files have been changed/edited/removed, and what static analysis violations map to those changes.
- The Code Coverage Explorer lets you navigate coverage from all of your testing practices, with traceability to the original tests that executed against the code.
- The Violation Explorer lets you view error reports such as MISRA rule violations in the context of the codebase, along with historical information, documentation, and specific details of the violation, so you can identify and trigger an issue remediation workflow.
3 — The Ways DTP Simplifies SDLC Workflows
This brings us to workflows. As you explore deeper into the data within the Explorers, they also provide the framework to help you take action. The DTP Explorers are connected through integrations to your existing software development tooling, so you can define actions for the team. For instance, you can create a task in JIRA, and augment that workflow by adding custom metadata to define things like priority, assignment, and additional context.
By connecting DTP to your workflow management systems, you’re able to take action at the same time as you learn what actions need to be taken.
But these explorers and resulting workflows are only as powerful as the data and analytics behind them. And here is where PIE comes in, which you may have heard of, either as a popular fruit-filled dessert, or as the award-winning Process Intelligence Engine (PIE for short), which calculates the advanced analytics and “derived metrics” that power DTP with true intelligence to inform your testing.
This brings us to number four.
4 — The Analytics that DTP Can Perform
DTP employs the Process Intelligence Engine (PIE) to provide actionable insights based on highly flexible analytics. DTP comes preconfigured with existing calculations called PIE “slices” (see what we did there..), which are intelligent analytics that empower your testing practices and subsequent decision-making.
Preconfigured PIE slices include:
- Test Stability Reporting - which helps you quickly identify and prioritize real regressions in the code vs. chasing instabilities.
- Change-Based Testing - which allows you to only re-test the tests that touched code changes.
- Modified Coverage - which lets you focus on the coverage of the code that’s changed, not the macro coverage that includes legacy code.
- Risky Code Changes - which keys you in on which changes in the code are more risky than others (and you can customize your definition of “risk”).
Using the pre-configured DTP PIE slices gives you exceptional power and insight on their own by providing key data that focuses the development team on essential work; however, the part that makes this so neat is that you can create your own slices of the pie with whatever analytics you can dream up. The framework is fully customizable, so you can extend it in any way you want.
5 — How DTP Helps with Compliance and Policy Enforcement
And so we reach number five, which isn’t necessarily the most interesting thing to talk about normally, but the value DTP provides in the form of automation and efficiency for compliance/policy enforcement makes this feature firmly take the number five spot on this list of how DTP goes "beyond."
For establishing and enforcing policies, DTP provides a devoted Policy Center, which helps you stay organized with practice thresholds, aggregated policies, and quality gates that can be queried as part of your CI pipeline.
On the compliance side of things, because of its aggregated data collection (see #1 above), with DTP you get a central location for managing and auditing your software testing in the context of whichever coding standards or rules you need to comply with. DTP provides three main useful things here:
- Compliance packs (for MISRA, CWE, etc.) that include preconfigured widgets for your dashboards and preconfigured reports to make compliance and audits a lot easier.
- The ability to take data from static analysis and display the results in context of your standard, without re-executing the analysis.
- The ability to, when a specific build is ready for certification, archive all the data from the test runs, locking the build once certification is complete.
So why has DTP been hiding in the closet?
Honestly, we sort of got wrapped up in how cool it is. Our developers created these awesome PIE slices that I described above, and we got so excited about the slices themselves, we forgot to tell you about what was actually happening behind the hood, that you could take advantage of.
So please, inquire. Get your hands dirty with this thing.
There is a treasure trove of value to be gained from Parasoft DTP. The analytic world is your dashboarding oyster.