Early last year we presented at 44con with a talk titled: “Penetration Testing considered harmful today“.
44con have just released the video so we figured it was worth a quick recap (for anyone not willing to tolerate the whiny voice!)
The original slides (in PDF) are available (here)
The central thesis of the talk is that penetration testing has established itself as a necessary activity for securing a network and is now pushed forward by a multi million dollar industry despite the clear signs that it is not helping all that much. (Read the annotated slides here)
Watch the video here:
10 comments On Penetration Testing considered harmful today
The timing was perfect! (thanks for the link)
Just read this, which i think is your point exactly!http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees/
Thanks DR. I think if it get us all thinking about it again, then it has served its purpose.
yes. Not sure who was saying “mission accomplished”. Not me. There were plenty who were saying that there is no need for tech analysis of risk any more because it suited their agenda. This is the real reason for the mess we're in today. Most issues can be traced back to this.re: boards. Growing numbers of them suspect problems because they see the lack of confidence in their CISO. They don't know what to do about it though. None of them have really been well advised to be honest.I agree pen testing, even under ideal conditions, should feature quite low in the whole strategy, but for different reasons. Not that i disagree with the reasons given, just I see it in a slightly different way.Companies can use zero days in testing but do it in a balanced way…e.g. first zero day compromises the listening service which doesn't give us root. then the tester may have a zero day for a setuid root binary..so, easy, game over. However, they could go about privilege escalation by other means so returning more value for the client paying the bill.I cover some of these issues in Chapter 7 of my book, Security De-engineering.good stuff re: extrusion. Overall, good presentation – refreshing aspects on pen testing.
One thing that is left out of the talk is the business 0-day issues that pentest simply does not have time to address. Any sufficiently curious internal user who has put time into learning internal environment can and will be superior in results to majority of pentest teams.
HiVery thought provoking presentation. I have been going back and forth with whether I agree or disagree with the article! Your presentation brings to the forefront a lot of problems (which needed to be done) but doesn't necessarily provide any great solution. The problem is of zero day's and your explanation and description really hits home the problem of just doing a standard pen test. However, information security is all about risk appetite and other great jazzy terms like risk tolerance and risk avoidance and risk acceptance. would it be too crazy to say that a lot of firms just accept the risk of 0-days and realise that if a hacker wanted to get to the crown jewels by either buying a 0-day or finding one themselves then they have accepted that risk. I guess one point I agree with you is that what use is the pentest nowdays? It probably is more a feel good factor for IT security managers, and at best a defense at script kiddies. Is that a great situation for IT security to be in, probably not.Another interesting point you bring up is browser vulnerabilities – these are definitely an issue, but I guess their place is not necessarily the pen test report. I guess it comes down to threat modelling again, and determining what, where are your major threats, and if one is a weakness in a browser then that needs to be classed as a threat and treated like any other threat.All in all, interesting article with a lot to think about.Good work.DR
Hi Jeff.Thanks. I'm pleased that it didn't take too much of a beating, because I genuinely think the dialogue is necessary.Thanks for the kind words.. :>
Great talk. Very thought provoking points. Not sure I agree with ALL of them, but many are absolutely valid. Thanks for raising the bar in our profession by promoting this dialogue.
Hi Andre, > The is a very great talk; a necessary talk; a talk that everyone in this field should hear.Thanks a lot, thats awesomely flattering.. I agree re: conclusions.. In truth the main thrust of the talk is to point out that we need a reset, and to try to justify why i think we do.I offer some (possible) solutions, but im fundamentally saying: “we need more ppl thinking about this””Trust, but verify” is gold and i think “TECHINT” will start making its way more and more into mature corps. All in all.. i think there are important conversations to have, and more deep thinking to be done..
The is a very great talk; a necessary talk; a talk that everyone in this field should hear.I find the conclusions in the talk to be unsatisfying (from my perspective at HP, which is where I work). I'm not going to say that more penetration-testing is necessary or even that you need to buy anyone's products per se, though. I'm not here to shill, but instead to attempt to say profound things.I think there is a lot more to be said about appsec risk management and penetration-testing-focused/oriented risk management, which is almost the point that you get to in your talk. In your view, it's a balance between a pen-test “Visio” and a playing-card game. Or worse, PTES. I'm not ready to jump to these conclusions without evidence, although I agree that it is likely more sound that what we already do in the pen-testing industry.My view of the “new” pen-testing industry is one of simplicity: “Trust, but Verify”. The best way to drive costs down in the verification process is to:1) Standardize on 'secure-by-default' application, data, and infrastructure components2) Penalize cutting/bleeding-edge and legacy (i.e. non-'secure-by-default') components by compartmentalizing/segregating them to a nearly non-functional state3) Reduce the number of components to the smallest possible amountVerification costs a lot of money, but it must be done. Another avenue of verification cost reduction is proper maintenance of coordinated disclosure through 'outside-bug-reporter-friendly' incident response teams (at all layers of arch: app and data, too!) and bug bounty programs.”Knowing is half the battle” — GI Joe wisdom that should be part of any information security and risk management program. Add the necessary intelligence. Add total intelligence, and especially add TECHINT (or technical intelligence, aka the study of one's adversaries' weapons and resources).Proper risk communication from subject-matter-experts to decision-makers should evolve modern information security and risk management programs to evidence-based conclusions. When it doesn't, then there is probably an issue with the core business decision-making process, which is often a greater risk to information assets than either one 0day or even a thousand.