Service Virtualization, Development Testing, API Testing Tips

Does Service Virtualization’s “Shift Left” Burden Developers?

Service virtualization undeniably benefits the development process, but it can be both a blessing and a curse for developers. Minimizing the burden that "shift left" can place on developers is key to achieving maximum acceleration of delivery cycles.

Service Virtualization's Shift Left Benefits the Development Process

Service virtualization’s potential to “shift left” testing is relatively well-accepted throughout the industry. With simulated test environments eliminating constraints that commonly delay or curtail testing efforts, testing can begin earlier. And, as we all know by now, the earlier you find a defect, the faster, easier, and cheaper it is to fix. Beyond that, service virtualization allows teams to test more extensively and more frequently (e.g., for continuous regression testing). 

Service virtualization’s shift left certainly yields significant benefits to the development process in terms of accelerating time to market, reducing risks, and lowering the costs associated with dev/test environment management. However, its impact on the actual development team is often overlooked.

But Does Service Virtualization Burden Developers?

In many respects, service virtualization is a gift to developers. First and foremost, it means their development and testing tasks aren’t stalled because they’re waiting for still-evolving components to be completed and/or staged test environments to be available. It allows them to create and modify “disposable” test environments on demand...without having to rely on someone else each time they need to tweak an existing configuration or access a new one. It relieves them from the minutiae involved in developing and managing effective stubs or mocks. It also enables them to access much more sophisticated behavior than stubbing or mocking can provide.

Yet, this “shift left” is not necessarily a panacea from the developers’ perspective. When you shift testing left, you also hasten the point at which QA is discovering and reporting the most defects. This means that instead of defect reports peaking during the testing phase, they peak at the development phase—which is when developers are already scrambling to implement the functionality needed to meet their development deadlines. 

Without Service Virtualization 

 

With Service Virtualization - The Shift Left

Getting barraged with defect reports during this critical phase is likely to cut into development’s time and focus on creating the innovative functionality that (you’re hoping) will set your organization apart from competition.

To understand what this shift left must feel like to developers, assume you’re expecting houseguests to arrive on Sunday evening, giving you an entire weekend to tidy up and prepare. Now, imagine that on Thursday evening they call to say they’ll be arriving Friday evening...and you have a major work deadline Friday afternoon.

So what do you do? Obviously, you don’t want to throw out the baby with the bathwater here. After all, service virtualization stands to deliver remarkable benefits and provide tremendous value to your organization as a whole.

Shift Left + Compress

The good news is that service virtualization doesn’t have to place extra burdens on development. The trick is to not only shift testing left, but also compress the defect curve. In other words, reduce the overall error injection rate so there are fewer defects to find and fix.

Shift Left + Compress

As you can see, this “shift left + compress” strategy avoids taxing development at their most critical juncture. Even though the defect curve peaks earlier, developers are not burdened with an increase in reported defects during construction time because the peak is lower. Moreover, because there are fewer defects to find and fix across the SDLC, the team is able to complete the entire iteration significantly earlier.

To return to our analogy, this is akin to your houseguests arriving early...but now they’re planning to stay in a hotel. Because you can focus on meeting your work deadline without worrying about cleaning, shopping, etc., the early arrival isn’t nearly as stressful.

How do you reduce the overall error injection rate? Through Development Testing: synchronized application of a broad spectrum of automated defect prevention and defect detection strategies in a way that reduces development risks, time, and costs. Depending on the organization's expectations and priorities, Development Testing might include static analysis, peer code reviews, unit testing, runtime error detection, and other software verification practices.

But isn’t this just placing a different burden on development? Not if it’s implemented smartly and unobtrusively. In fact, Development Testing can actually improve productivity while reducing risks. But that’s the topic for another blog…

Service Virtualization ROI Paper 

Curious about what ROI your organization can achieve with service virtualization? Read Parasoft’s new 5-page Service Virtualization ROI white paper to learn about the business drivers behind service virtualization purchase decisions, as well as the substantial opportunities for ROI in terms of OpEx reduction, CapEx reduction, risk reduction, and incremental top-line revenue.

How Mature is Your API Testing Process?

Learn Where You Stand Today and How to Reach the Optimized API Testing Level

Without an enterprise-level automated solution for ensuring the integrity of APIs and API-driven composite applications, organizations risk:

• Brand erosion as faulty software drives away customers
• Time-to-market delays that diminish market share
• Exposure to legal liability associated with application failure
• Failure to comply with applicable regulatory standards and technical contracts

API Testing solutions help organizations reduce the risks, costs, and resources associated with exposing and consuming APIs. The application of API Testing solutions can range from very simple ad-hoc or reactive efforts to highly-complex test environments driven by business risks.

Ad-Hoc API Testing

With ad-hoc API testing efforts, no formal process or tool is used to unit test or exercise the API. It is assumed that the API is exercised via manual testing of the UI. Ad-hoc API testing characteristics include:

• Organization has invested little in test automation. 
• Manual test efforts are the primary driver for QA. 
• Defects are commonly found in deployed applications. 
• Test breadth is severely hampered by lack of automation.
• Limited understanding of the dependent endpoints.

Any pockets of maturity at this point are based on the experience and initiative of individuals. There is no centralization of assets; it’s every man for himself. Along the same vein, testing assets are typically created as one-off solutions and stored on a local machine, inaccessible to anyone but the creator. There’s no test automation here; it’s all manual, ad-hoc execution.

Organizations are driven to move to a mature level of API testing when:

• Proliferation of APIs exposes weakness within the quality process, requiring an API-centric view for testing.
• Brittle manual tests impede agility. 
• Composite applications with dependencies beyond the group’s direct control add complexity.

Optimized API Testing

With optimized API testing efforts, business risks drive the testing process and the optimization of associated policies. Optimized API Testing characteristics include:

• Optimized environment for goal-oriented, business-driven scenarios significantly reduces application risk.
• Test scenarios are reused as components of complex end-to-end transactions.
• Consistent, continuous environment access enables more extensive and accurate testing to occur with or without access to a staged test environment.
• A Center of Excellence is established to optimize and manage policies, procedures, and standards.

At this point, there’s seamless integration and orchestration of Service Virtualization with virtual test lab management systems. Automated regression suites are called and executed against complex environments and environment-based views deliver perspective on coverage and business risk.

API Testing Maturity Model

This is just a brief introduction to the two extremes of API testing maturity. Most organizations today fall somewhere in between these two polar extremes.

Parasoft has developed an API Testing maturity model that provides a detailed look at the 5 different levels of API testing: Ad-hoc, Reactive, Proactive, Managed, and Optimized.

If you want to assess where your organization currently stands and see what’s involved in moving forward, download the complete API Testing Maturity Model.

Static Analysis & Development Testing for Embedded Devices

By Jason Schadewald, Product Manager at Parasoft

You know those conversations that you have more times than you can count? Well, I recently had one of those at Design West with a very bright software engineer. This poor guy had a number of experiences with static analysis tools that left him with the “compiler warning equivalence” impression. If your static analysis experience is largely with freeware and your training is limited to Internet forums, then I certainly understand how that impression can form. On top of that, he said that the static analysis tools he tried reported “over 20,000 messages.”

It’s easy to see why he and many developers like him would find the effort insurmountable. What we’re dealing with here is a question of validity and quantity of results, and a mature Development Testing platform will help you manage both with minimal human intervention.

Validity of Static Analysis Results

Medical, automotive, aerospace, railway, and indeed all software developers struggle with the concept of validity of results on a daily basis, and the objects of scrutiny range from customers to QA to management to the very tools on the desktop. For developers, it can sometimes seem like everyone and everything is a source of new work – and it all has to be done yesterday. (Is it any wonder that the average developer stays with a company for only 2-3 years?)

While validity is a multifaceted topic, this guy was focused specifically on the “warning” nature of the messages. If it’s just a warning, then it’s not an error, and it’s not worth his time.

What’s really going on is that many development testing tools fail to justify their results in an easily accessible manner. Some even try to hide it behind a glorified “triage” process! In reality, different static analysis rules serve different purposes – defect prevention, consistency for productivity, actual bug-finding, etc. A mature tool will provide full justification for its results as well as a means to organize them by priority, severity, and risk.

This brings us to the next point...

Quantity of Static Analysis Results

No self-respecting manager would drop 20,000 tasks in his employee’s lap, so why do we accept static analysis tools that do? A well-implemented development testing strategy will allow for the prioritization of tasks by severity and risk. Additionally, it will account for distribution of workload across the team and take advantage of individual strengths.

As I confirmed with my new friend from the convention, he would have no problem fixing 5 static analysis violations per day. If he has a team of 10, and they each take 5 static analysis violations per day, then he’s looking at sparkling clean code in about 400 days.

Let’s take this a step further, though. Not all static analysis violations are created equal. If we frontload that effort with the highest risk, most severe issues, then quality, safety, and security can be reasonably addressed within 1-2 months. That’s less than one release cycle for many shops – and with negligible impact to schedule at 5 minutes per task.

The icing on the cake, of course, is that a mature development testing platform will automatically manage the prioritization and distribution of those static analysis tasks for you. You, the manager, set a few policies at the outset, and those policies ensure that each member of your team gets a reasonable number of the highest priority tasks each day. Everybody wins.

 ***

For Development Testing white papers, articles, and videos/webinars, visit our Development Testing Resource Center or watch the brief video below:

 

For Static Analysis white papers, articles, and videos/webinars, visit our Static Analysis Resource Center. 

Security Has Landed in the World of Embedded Software Testing

By Jason Schadewald, Product Manager at Parasoft

I’m glad to see that security has finally landed as a hot topic in the world of embedded devices, especially given that—only a few years ago—the mention of security in an embedded context would earn you some good laughs.

At Design West last week, I noticed several vendors— including a number of Parasoft’s partners—in hardware, firmware, and software showed up with solid security solutions.  While the specifics varied, there were two important and common themes that reflect the lessons learned years ago by the web and IT industries:

• Security is everyone's job
• Build security in

Security is Everyone’s Job

In the world of the web, application developers left security concerns to the OS, to network and message encryption, and to IT firewalls or OS user access controls.  When an attack exploited the application itself, developers and executives alike lost their jobs. 

The lesson for the embedded community is that every one of us needs to be security conscious.  The Internet of Things makes every device vulnerable to the same (or slightly tweaked) attacks that are all too familiar in the world of web applications.  Add to that the potential of an attacker gaining physical access to the device, and suddenly the number of security considerations has multiplied.  

Build Security In

There are two types of secure applications: the ridiculously simple, and the ones that considered security from start to finish. 

At several booths, I heard a similar story: Organizations did security testing, but it obviously wasn’t enough because their device was compromised—sometimes for the data, and sometimes just as a means to get to another system. In this situation, “more testing” will never be enough. 

As someone who flies in your plane, drives your car, trusts his identity to your phone app, and might one day find himself hooked up to your medical device, I implore you to remember that your application is a critical piece of a larger security infrastructure and that you need to design for security from the very beginning.

 

***

Want to learn more about embedded testing and secure application development strategies and best practices? Visit Parasoft's embedded resource center to view videos, articles, papers, and more. 

How to Spring Clean Your Static Analysis: 10 Tips

Is your development team overwhelmed by the mounting number of violations from your static analysis tool? Has the high level of noise being generated by your current static analysis configuration desensitized the team to all alerts—including those for issues that you consider critical?

Now that we're in the middle of spring, it's the perfect opportunity to revitalize your static analysis efforts with a little spring cleaning. Start off by clearing out the clutter that's making it difficult to zero in on the issues you truly care about. Next, invigorate your initiative by expanding its scope in ways that increase its value to your organization.

Here are 10 ways to freshen up your existing static analysis implementation—no matter what static analysis tool you're using.

Tip 1: Disable static analysis rules for violations you're not committed to fixing right now

Checking a lot of rules is not the secret to achieving the best ROI with static analysis. In fact, in many cases, the reverse is true. Static analysis actually delivers better results if you focus on a minimal yet meaningful set of rules.

When you perform static analysis, it's like you're having an experienced developer stand over the shoulder of an inexperienced developer and give him tips as he writes code. If the experienced developer is constantly harping on nitpicky issues in every few lines of code, the junior developer will soon become overwhelmed and start filtering out all advice—good and bad. However, if the experienced developer focuses on one or two issues that he knows are likely to cause serious problems, the junior developer is much more likely to remember what advice he was given, start writing better code, and actually appreciate receiving this kind of feedback.

It's the same for static analysis. If you keep it manageable and meaningful, you'll end up teaching your developers more and having them resent the process much less. Would you rather have a small set of rules that are followed or a large set that is not? If you don't truly expect the developers to clean violations of a rule as soon as they are reported, you might want to seriously consider disabling that rule.

Tip 2: Disable static analysis rules causing too much noise

If a particular rule is being violated repeatedly, now is a good opportunity to re-evaluate whether you really want to continue checking that rule. An excessive number of violations indicates that the developers are not writing code in the way that the rule requires. Convincing them to change their coding habits could meet a fair amount of resistance.

How can you determine if pressing the issue will be worth the effort? First, try to remember why you started checking for that problem in the first place. Did you select it because it seemed like a good way to address issues you're experiencing in the field? As part of your regulatory compliance efforts? Or simply because it was enabled by default by the vendor? Vendors typically provide the reference for each rule in their rule descriptions. Reading these descriptions can help you determine if the rule is really a good match for your projects and goals.

Next, see if there's an alternative way to achieve the desired result. Is there an alternative rule that's more specific? Is there a way to fine-tune the rule parameters so that it's not firing so often? (More on this in tip#6). You might even consider writing your own rule that will be a better fit— or have the vendor create a custom rule for you.

If you're still interested in checking this rule after re-examining its benefits and exploring its alternatives, get some development feedback on what would be involved in following this rule. You can then use this feedback to determine if it's truly worth it to require developers to follow this rule. If it looks like a lot of work for little benefit, go ahead and disable the rule.

Tip 3: Use suppressions to allow violations in in specific situations

In some cases, you might be committed to following a rule, but want to allow exemptions under certain circumstances. For example, maybe you have a rule that requires some extra level of validation to be performed in the code. Assume you have a certain method with performance-critical code that is called hundreds of times a minute—and you've already verified that an appropriate level of validation is performed before this particular method is called. Or, assume that flow-based analysis is warning you about a serious problem in a path that you are 100% certain cannot be taken in the integrated application. This is where suppressions come in handy.

Suppressions are ideal for situations when you want to check for something, but you don’t care about reported problems under exceptional situations. With suppressions, you can continue checking for a critical issue without receiving repeated messages about your intentional rule violations. If you do not want to receive error messages for any violations of a specific rule, we recom¬mend that you disable the rule altogether (see point 1).

You can typically define suppressions from the static analysis tool GUI, a configuration file, or the source code itself. When suppressions are defined in source code:

• You ensure that the same suppressions are applied whenever you or a team member analyzes that code.

• You can add code comments explaining each suppression so the reason for each suppression is always clear when you or team members are reviewing or modifying the code.

• You gain fine-grained control over which rules are enforced at the file, class, or line level. 

You can typically suppress violations of a specific rule, a number of rules, or all violations in a specific category. You could also exempt certain sections of code from all static analysis (more on this in the following point).

Tip 4: Stop analyzing problematic files or chunks of code

Sometimes it just doesn’t make sense to run static analysis on certain files—for instance, automatically-generated files or legacy files that you don't plan on touching. In these cases, you should prevent these files from being analyzed. This is yet another way to ensure that your results aren't cluttered with a bunch of violations you're not planning on fixing. 
There are a few ways to do this. You could set up path filters to exclude files you don't want to check or include only the ones you do want to check. Or, you could configure your tool to skip files that contain a certain comment—such as a comment indicating automatically-generated code.

Other ways to focus your checking include:

• Adding markers to indicate specific blocks of code you want checked within otherwise exempted files.

• Excluding certain methods within files that are otherwise analyzed. 

• Checking only files that were not added or modified since a certain cutoff date.

• Checking only files that were added or modified after a "cutoff date" or within a certain number of days.

Tip 5: Notify the static analysis tool vendor of broken rules causing false positives

With pattern-based static analysis, false positives are rule violations that are erroneously reported when the code actually follows the rule. For example, if the rule says you have an unclosed resource (such as a JDBC connection), when in reality the connection is closed, then this is a false positive. If you encounter an issue like this on a rule that you really want to follow, spring cleaning is a great time to finally report it to your vendor.

Note that if you’re going down this path, you need to be certain that you’re looking at an actual false positive, rather than simply a rule that you don’t like. Developers frequent call a message a "false positive" because they don’t like the rule, or don’t feel it applies in this instance. Such messages are not false positives and your vendor will not be able to help you in these cases.

However, if you can reduce a simple test case that shows how a particular rule is actually getting a false message, you should find most vendors are very helpful in remediating the problem.

Tip 6: Customize static analyisis rule parameters to suit your needs

Another way to reduce the noise factor is to customize rule parameters. For example, assume that your team is doing Android development and you're checking an Android rule that says "Make sure that widgets aren't updated too often."

With the default settings, this rule identifies code that sets a widget to update more than 4 times per day. It does this by flagging code that sets the element [android:updatePeriodMillis] in the tag [appwidget-provider] to a number smaller than 216,00,000.

Assume that getting updated information is critical to your application, so you're willing to sacrifice some battery consumption for more frequent updates. In this case, you might want to be warned only if updates occur more than 8 times per day. To achieve this, you could simply update the "Maximum update time maximum in milliseconds" rule parameter from 21,600,000 ms (6 hours) to 10,800,000 ms (3 hours).

As tip #2 mentions, if the rule doesn’t have the parameters that you need, you can write a new one that does—or have your vendor (or a consultant) write a custom rule for you.

Tip 7: Map static analysis rules to your own terms

Are you tired of remembering that Security 123 is equivalent to your ACME 3.1 guideline? That both Performance 987 and Performance 567 are related to your ACME 5.6 guideline? That even though your tool says Threads 123 is severity 4, your organization considers violations of that rule to be a very severe defect? 

If so, spring cleaning is a great time to map your vendor's static analysis rule set to match the distinct policies defined by your team and/or organization. Most static analysis tools let you customize rule severities, IDs and names—as well as create new rule categories—so that the deployed rules precisely match the contents of your own coding policy document.

If your organization performs static analysis as part of a compliance effort, this will make your reporting a lot easier.

Tip 8: Re-examine and clarify your static analysis policy

Every team has a policy, whether or not it's formally defined. You might as well codify the process and make it official. After all, it's a lot easier to identify and diagnose problems with a formalized policy than an unwritten one.

Ideally, you want your policy to have a direct correlation to the problems you're currently experiencing (and/or committed to preventing). This way, there's a good rationale behind both the general policy and the specific ways that it's implemented.

With these goals in mind, the policy should clarify:

• What teams need to perform static analysis

• What projects require static analysis

• What rules are required

• What degree of compliance is required

• When suppressions are allowed

• When violations in legacy code need to be fixed 

• Whether you ship code with static analysis violations

Tip 9: Increase the scope of analysis

Once you've cleaned out the clutter and you're at the point where the team is used to performing static analysis, few issues are being reported, and those reported issues are being cleaned up promptly, you can take the next step and expand the scope of checking.

One way to expand the scope of checking is to add in more rules that are critical to your projects and goals. To zero in on what rules to add, consider:

• What rules seem likely to prevent field-reported issues that are consuming a lot of your team's resources?

• If you'll soon need to start complying with industry-specific or organizational compliance initiatives, what rules would help you jumpstart your efforts?

• When you first created or last optimized your rule set, what rules were you most reluctant to cut? 

Another way to increase the scope of checking is to check additional code. If you initially set your static analysis tool to skip legacy files (e.g., skip any files that were not added or modified after the "cutoff" date when you began static analysis), you might want to consider moving back that cutoff date—or eliminating it altogether.

Tip 10: Automate, automate, automate

The more you can automate the tedious static analysis process, the less it will burden developers and distract them from the more challenging tasks they truly enjoy. Plus, the added automation will help you achieve consistent results across the team and organization.

Many organizations follow a multi-level automated process. Each day, as the developer works on code in the IDE, he or she can run analysis on demand—or configure an automated analysis to run continuously in the background (like spell check does). Developers clean these violations before adding new or modified code to source control.

Then, a server-based process double checks that the checked in code base is clean. This analysis can run as part of continuous integration, on a nightly basis, etc. to make sure nothing slipped through the cracks. With such a server-based process, it's important to avoid the old paradigm of sending email to developers. Part of an effective workflow is distributing the error messages to the same UI where the developer writes code. Email forces extra steps, leading to missed violations, time wasted finding the proper line in the file, and more resentment from coders who feel like they’re doing something extra outside of their regular process.

To further optimize the workflow through automation, consider:

• Automatically routing each reported issue directly to the responsible developer—as well as customizing issue prioritization to suit your policy priorities—to ensure that your most critical issues are addressed in a timely manner.

• Centralizing configuration management to ensure that rule sets are applied consistently and can be updated effortlessly as priorities and processes evolve.

• Leveraging automated "Quick Fix" refactoring whenever feasible to help the team correct rule violations as fast as possible.

 

Top 10 mistakes with static analysis

All too often, teams flirt with static analysis for a few months or a year, but never truly commit to it for the long term. This is a shame because static analysis, when properly implemented, is a very powerful tool for eliminating defects—with minimal additional development effort.

Read Parasoft's Top 10 Mistakes with Static Analysis eBook to see the top 10 reasons why static analysis initiatives don’t deliver real value—and tips for avoiding these common pitfalls. Or, visit our Static Analysis Resource Center to browse videos, articles, and papers about static analysis. 

The Static Analysis Spectrum: 5 Types of Static Code Analysis

Static analysis is a Development Testing activity that can drive a software development team's productivity and minimize fiscal, legal, and ethical risks associated with potentially faulty code.

The reason many organizations fail to fully realize the benefits of static analysis, however, is that it's often homogeneously deployed as a tool for "finding bugs." But the truth is that there are different implementations of static analysis that serve different purposes in the development process. And while it's a foregone conclusion that developers should run static code analysis, the proper implementation of the right type of static analysis can make the difference between wasting time and money and delivering better software faster.

Integration-Time Static Analysis Definition

Running static analysis during integration to detect low-hanging fruit and egregious errors is a good starting point for implementing a preventive strategy. Integration-time static analysis simulates feasible application paths without actually executing the code, which is very helpful for systems in which runtime analysis isn’t possible. Static analysis can test across multiple functions and files and catch common memory problems, such as uninitialized memory, overflows, null pointers, and so on.

Static analysis serves a few purposes in terms of the development strategy when organizations begin with testing during integration. First, developers can review the test results and determine how important they are for the particular application. Static analysis might uncover potential defects that might have a serious impact on software security, reliability, or performance. On the other hand, it could return things that the business might not care about. For example, business probably doesn’t care about a defect in a gaming console that causes the software to crash when an unlikely sequence of operations occurs. The user can simply reboot and continue enjoying their system. Resolving the same sort of issue in other contexts, however, might be crucial to preventing catastrophic consequences.

Static analysis can also help developers find potential defects that would have been very difficult to conceive of during the risk assessment phase. Developers can catalog potential defects to improve future risk assessment iterations.

Continuous Integration-Time (CI) Static Analysis Definition

After running integration-time static analysis, developers should have a stronger sense of potential systemic problems in the code. The next step is to run CI static analysis to enforce the coding policy outlined in the planning phase. This prevents the types of defects discovered during integration-time analysis.

For every issue discovered in static analysis, there are at least 10 more of the exact same thing in other places in the code. Static analysis is the ideal tool for addressing all violations of the same kind at the same time. This is opposed to chasing every possible path through the code. It’s far better to find the systemic problems in order to create an environment in which bugs cannot survive.

When we talk about static analysis, in many cases we mean anti-pattern analysis. A positive pattern is something that should be in the code. For example, a policy that requires developers to use a typedef when declaring function pointers is a positive pattern static analysis rule[1]. This is in contrast to a policy that, for example, prohibits the use of the data() member function from a string class when interfacing with the standard C library[2].

Executing both types (positive- and anti-pattern) of static analysis is important, but it’s worth mentioning this distinction because if the organization spends the time to build a coding policy based on positive patterns, this ensures that developers are building code exactly how it should be per business objectives or compliance requirements.

Metrics Analysis Definition

Metrics analysis is a static analysis implementation that evaluates code characteristics and provides insight about the code that can help developers identify weaknesses (Figure 1). It is a critical sensor that can highlight areas of the application that can be prone to logical errors. Metrics analysis is an essential baseline measurement that should trigger further analysis, such as code review or some other remediation activity.

 

A Parasoft static analysis metrics report

 

Metrics analysis is best used as early as possible because it might affect how developers write their code. Avoid trying to implement metrics analysis reactively or during the QA phase. The goal with metrics analysis isn’t just to detect potential defects; it’s to detect them in such a way that allows developers to follow a sustainable coding trajectory. Run metrics analysis on potential defect hotspots, remediate any violations, and implement a pattern-based analysis rule to prevent future occurrences.

Any metric that correlates to potential problems is fair game. For example, a medical device company might use metrics analysis to measure cyclomatic complexity because a high score indicates that there are too many decision points for the device to handle during normal operation. Knowing that the complexity score exceeds the threshold set in the coding policy when there are 10 branches to cut, as opposed to finding out in the QA phase, will help keep the project on time and on budget. The organization might, for example, want to measure public variables because a high number might correlate to too many dependencies in the code. Each organization will need to decide which metrics can be correlated to possible defects in the code.

Edit-Time Static Analysis Definition

The static analysis sweet spot is while the developer is working in the editor. Running static analysis at edit time serves a few purposes. First, it points developers to potential problems. Second, it implements the risk assessment strategy by ensuring that any issues are remediated systemically.

But when should static analysis be implemented? We’ve discussed why implementing static analysis too late is a problem; however, it can also be implemented too early because there must be enough context for static analysis to provide meaningful information. Running static analysis on a character, line, or even statement creates too much noise to be useful. Enforcing positive design patterns ensures that new code is built as intended – while it’s being written. Running static analysis at edit time is a powerful way to promote the correct behaviors within the development team because feedback is rapid and in context of the code being written. Leveraging this type of analysis makes code reviews more productive because developers should be able to correct policy-based errors immediately.

Runtime Static Analysis Definition

Some static analysis patterns can detect defects at runtime. If the embedded target can accommodate the overhead, the organization should execute runtime static analysis to round out its preventive strategy. Runtime static analysis detects errors while the code is actually running, which enables developers to test real paths with real data.

***

The Top 10 Mistakes with Static Analysis eBook

All too often, teams flirt with static analysis for a few months or a year, but never truly commit to it for the long term. This is a shame because static analysis, when properly implemented, is a very powerful tool for eliminating defects—with minimal additional development effort.

Read Parasoft's Top 10 Mistakes with Static Analysis eBook to see the top 10 reasons why static analysis initiatives don’t deliver real value—and tips for avoiding these common pitfalls. Or, visit our Static Analysis Resource Center to browse videos, articles, and papers about static analysis.

Exposing Your APIs = Opening Your Home to Bears?

Exposing an API to your application is as risky as installing a doggie door into your house: you expect your dog to have the convenience of outdoor access, but the reality is that you cut a hole in your house. 

The intended use is to give the anticipated “users” access to an area that is otherwise private. However, once you open that portal into your sanctuary, there’s no telling who else might take advantage of it. Just as the doggie door can open your home to all sorts of wildlife, so can API exposure open the door to application usage you never anticipated (both innocent misuse and malicious attacks).  

A recent incident in Parasoft's home town of Monrovia, CA  provides a real-life example of how a doggie door can lead to some rather unexpected consequences... 

A Monrovia resident recently came home to find that 2 bears entered his home through a doggie door, ransacked his kitchen, and ate everything in site. For the full story, including some video, see Bears Break Through Pet Door, Ransack Monrovia Kitchen.

 

API Testing Challenges and Best Practices

As organizations expose their business-critical services as APIs, test and QA teams need to ensure the organization is protected against the threats and dangers that such exposure could bring.

Read Parasoft's new white paper on API Testing Challenges and Best Practices to learn about the top 4 API testing challenges and 5 API testing must-haves for addressing these challenges.

 

Service Virtualization Definition

What Is Service Virtualization & Why Does it Matter?

A recent blog by Hurwitz & Associates analyst Marcia Kaufman defines service virtualization:

“Service virtualization is used to simulate the behavior of components in an application so you can perform an accurate and timely test in a world of complex interrelated applications. Production services that may not be available for integration testing can be virtualized so the testing can take place at an appropriate time in the software development process.”

Kaufman also explores the benefits of service virtualization:

“One of the biggest impacts of service virtualization for developers is the ability to validate integrations much earlier in the application life cycle. The software development team can move beyond unit testing and overcome many of the roadblocks that inhibit timely, efficient, and cost effective testing.”

We strongly encourage you to read her complete What is Service Virtualization? article. After exploring why service virtualization is so critical given current methods for writing and combining code, Kaufman presents 5 key things to know about service virtualization. These service virtualization considerations can briefly be summarized as:

1. Consider how service virtualization can complement your testing methodology.
2. Use a cost/benefit analysis to determine which services are prime candidates for service virtualization.
3. Recognize that service virtualization can help you detect errors during all testing phases.
4. You can define a service virtualization virtual asset by recording the behavior of an existing component.
5. Plan to rapidly shift back and forth between real and “virtualized” components when you’re testing.

Service Virtualization ROI Paper

Curious about what ROI your organization can get from service virtualization? Read Parasoft’s new 5-page Service Virtualization ROI white paper.

15 Seconds to Find a Pointer Error In 500,000+ Lines of Code

Reducing the Length and Cost of Downstream Development Processes

For over a week, Okrey tried to re-engineer different pieces of the class that was causing trouble.

But his attempts at fixing the  problem only resulted in changing some of the  internals so that the point of failure occurred in a different location. “I spent more than 40 hours going through all of my code  with a fine tooth comb  and a magnifying glass, like I usually do. I was unable to locate the problem. I could see what was happening; I just could not see why it was happening,” Okrey said.

That’s when his search for help began. He found only a handful  of tools that were able to do what he wanted. Of that handful, most of the products merely allowed for static review of code. Parasoft was the only product that also performed dynamic analysis. “Parasoft gives me the ability to analyze my content in the environment  where it’s being run as opposed to just looking at the code on paper,” Okrey said.

After getting  set  up  and  running, Parasoft ran  through the  first  build  of Okrey’s source code—all 500,000+ lines. Within 15 seconds of launching, Parasoft surfaced a stale pointer error. “If I hadn’t found Parasoft, it would have led to very drastic requirements from the client,” Okrey said as he reflected on the rapid return on investment. He added that “To go from a functioning version of the program to a non-functioning version simply due to an upgrade would have led to a reversal of progress and forced financial  concessions that  I do not want  to even  consider. It was an ugly situation.”

 

Increasing Code Quality, Stability, and Compliance

Parasoft enabled Okrey to completely revamp the  toolkit  source code;  specifically,  improving string-handling. The improvement spread to other projects. Okrey has dozens of other programs for various clients  that  use  the  same backend toolkit, so all of these programs reaped the  benefits. Okrey states, “I can’t even begin to tell you all of the programs that are dependent on the backend toolkit. As a result of the improvements Parasoft enabled me to make, they are all that much more stable and compliant.”

Okrey said that the Parasoft Development Testing Platform gave him the ability to implement and enforce his high coding standards. “Parasoft forces you to verify that the standards and practices that are  being  used are  absolutely pristine,” Okrey  said.  “One of the  challenges as  a project leader—or managing partner, like myself—is confirming that your team is writing code that meets high standards. Parasoft can help me verify that my team is writing code that meets my standards and allow me to guarantee results. I am really excited about that.”

 

Finding Value in Parasoft’s Development Testing Platform

Okrey is pleased with the quality that Parasoft has rapidly ingrained into his application development process. Not only has he been able to rectify a problem for a valued customer, but he has also been able to improve the quality of dozens of programs for other customers.

Okrey says, “I’m very particular  about products that I choose to endorse. The majority  of software written  in the world just doesn’t work the way it’s supposed to work for various reasons. Maybe it’s poorly designed so it runs slow, or system requirements aren’t realistic. The list goes on.

“However, there are a few products that I really like. One of those is a system software product that I’ve come to rely on. I’ve never experienced a GPF with it. Never. When I learned that the provider of that product was a Parasoft customer, that was it. That’s what made me decide to give Parasoft a try and I’m very happy that I did.”

Development Testing Podcast: Do You Know What NOT to Test?

Success In Building Quality Software Is Knowing What Not To Test

In the latest Techpost Media podcast, Parasoft Chief Evangelist Arthur Hicken explores how “test-driven” development (TDD) and Agile bring useful things to the software engineering, but also have some impractical aspects. The trick is to use what is helpful and discard the rest.

Listen to the 20-minute podcast "Success In Building Quality Software Is Knowing What Not To Test" to explore how software testing has evolved over time, predict where it's heading in the next 5 years, and get tips on how to:

• Better leverage test automation to keep up with the pace of Agile and help the team focus on tasks that truly require human intelligence.

• Focus on testing only those things that are truly critical to verify quality.

• Keep testing noise at a tolerable level.

***

Want to learn more about quality strategies for software development? Read the Development Testing wikipedia article or browse resources at the Parasoft Development Testing Resource Center.