Pete Finnigan's Oracle Security Weblog

I made a note a couple of weeks back to talk about this subject here when I got a chance. Well that time has arrived. I saw a post on the pen-test list on security focus titled "Research on penetration testing?" where the poster told us that he had the chance to take a masters degree in Norway and was looking for research ideas in the area of penetration testing, unanswered questions, areas that should be researched further academically or ideas that would be of public interest.

A few posters suggested ideas, the complete thread index is available here. One poster suggested looking at the business aspects of costing and benefit to the customer of penetration testing. I thought that this was a great idea. I added a detailed post with my thoughts at the time. I will repeat my ideas here as posted:

"I would agree with this idea. When I read this my thoughts went
immediately to the book "Optimizing Oracle performance" written by Cary
Millsap and Jeff Holt - Published by O'Reilly. This book, quite
obviously by the title is about tuning Oracle and not about penetration
testing - bear with me..:-)

The book describes the new (ish) method of using the Oracle wait
interface (instrumentation in the Oracle kernel) for tuning. But the big
idea in the book is that fact that using this method the tuning effort
is repeatable and calculable in advance in terms of effort and cost
benefits.

Cary and Jeff describe how its possible to analyse the issue and then
identify the key business processes that are a problem and finally also
identify the time saving that is possible with solutions (This is
possible because what they are doing is analysing lost time in the
processing of data - so that time is highlighted in detailed steps in
the kernel source) and hence the cost benefit to the organisation.

This is mind blowing when you consider previous efforts were based on
trial and error. e.g. change this parameter and see if the program is
faster... no... now change it back and then try another parameter
instead... and so on.... with "method R" as described by the authors the
tuning effort is acutely focused based on cost benefit to the business
and the cost benefits of the possible solutions are known.

Now if this breakthrough in tuning could be applied to penetration
testing then the cost benefits for customers would be great. I would say
also that anyone who could offer this service would be in a commanding
position. Managers like to see costs and benefits..:-)

It is like the difference between alchemy and science.

Whether its possible to apply these ideas to penetration testing or not
is difficult to know. Also I feel the solution would probably be
technical as well as business related. In the Oracle book, the authors
have developed perl scripts to apply the ideas and also utilize queuing
theory in the solutions. I think ideas along this line would make a
great project.

Hope this helps"

I think this is a very good point in relation to general penetration testing. How would you define the cost benefits of a penetration test and how does that relate to an audit of to internally running a commercial or free tool?