By Edd Hardy, Head of Operations, CNS Hut3

For large businesses the traditional once-a-year security test, or penetration test, has become at best problematic and at worst unmanageable. Unwieldy spreadsheets and a lack of communication between the business and IT are leaving organisations with exposed business-critical information, while resource is spent on ineffectual fixes. In short, the traditional pen-testing model is no longer an effective tool for handling the complexities of larger organisations. How has this happened and how can organisations improve it?

This has become a particular problem for large companies because of the way IT has increased in complexity over the years. Big organisations now have vast IP address ranges, they have a huge volume of services and complex systems exposed to the internet and they have a distributed IT structure. Very often different people are responsible for different bits of different systems in different countries.

Added to this are the improvements in testing over the years and the automated tools which can find many more issues. This means that, for large organisations, a security check or penetration test no longer produces a 50-page PDF of issues, it can run to thousands of pages.

Given the volume of data about security issues, it’s no wonder that large companies struggle with prioritising critical issues. Even if a business can afford to fix them all, which do they fix first? How can they tell which leave them more vulnerable? Often the pen test doesn’t help, because the tester’s opinion of what is critical (perhaps a denial of service attack) doesn’t reflect that of the business, which might be much more concerned about a loss of customer information or credit card details. Too many penetration tests fail to work with the business to prioritise the resulting data.

Penetration testers see the results of this lack of useful information. Standard operating procedure is that an issue is found and reported to the client who fixes it. The security company then confirms the issue is resolved, only to find that the next year, when they come to retest, the same problem has returned. From a security and ROI point of view this is worrying. It means all we have achieved is getting rid of the vulnerability for a relatively short period of time. In addition, we are not even able to say how long the vulnerability went on for.

So how can we create useful information out of the penetration test data? What can IT and business do to prevent irrelevant security fixes and rediscovering the same old problems? We think the industry should be presenting information, not data and this can be achieved by adding four critical steps to penetration testing operating proceedures: weighting issues, tracking remediation, resolving root causes and reporting information (not just data). This can all be done by:

Weighting the results

We find too many issues, organisations need a way to zoom in on the issues that matter to them. By engaging the customer before testing and identifying what really matters to them and the risks that worry them, we can help customers focus on the issues that matter to them, not the ones the penetration testers find interesting.

Tracking improvements

It is very hard for organisations to prove they are getting better, every year the testers find more issues. By tracking and continually testing environments, it is possible for clients to show they are reducing risk, not just fixing issues.

Look for the root cause

Organisations do not take security lightly. They want to be secure and to be able to demonstrate it. So they make a huge effort to fix issues, but often can miss the root cause. They are treating the symptom not the disease. We see this as issues are fixed but come back next year. Testing companies should be helping organisations to locate the route cause. This can be as simple as correlating the data and showing that a few days fixing a broken process is a better investment than a few days fixing individual issues.

Present information not data

Cyber security companies need to present the results of penetration testing in such a way, that an organisation can find commonalities, which will help create more efficient ways to solve the problem. At the moment the industry is giving customers a vast data set and expecting them to make use of it. By providing information not just data, large organisations will be able to find the patterns and work out what to fix, how to fix it, and track the improvements, which will all help to demonstrate the ROI of a security check.

As penetration testers we want to find the new and exciting stuff, not repair the same issues every year. We think this means mending the penetration test system and creating reports which generate manageable information, not just data.