Teach, don't scold
Kennedy said he advises pen testers using the kit prepare the company in advance that the success rate of the pen test is likely to be high. But even with some warning, that may not be welcome news to the organization. This can be a chance to teach them, rather than point out problems, said Kennedy.
"One thing that drives me nuts in security community is the rating of users. Somehow it is supposed to be the user that understands this stuff. But for those of us in security, it is our job to teach and not to scold. When you do these kinds of engagements, it is an education opportunity, not a "you-did-something-wrong" opportunity."
Kennedy recommends letting the organization know that when a user makes a mistake and falls for a social engineering scam, this is perfectly acceptable and happens to everybody.
"Tell them: "This is something we learn from, and here is why it was bad," and point out some things they can learn from in the future," he said.
Also assure them the likelihood of a better score in future pen tests using the kit is likely.
"Users will start to recognize these things with repetition," he said.
Embarrassing a company due to its flaws is a horrible idea, said Hadnagy.
"Any time an audit is done the results should be used as part of employee education. This can be done without embarrassment by educating the employees first at point of failure. For example, when I do phishing for my clients, I do not just include their names in the report, but when the employee clicks they are automatically sent to an education page about phishing."
Hadnagy said during the mass education an employee should not be mentioned and no jokes should be made. A tool like SET allows a tester to track who clicks and who responds, this can be beneficial because in larger organizations it can point out areas of weakness and where education can be more beneficial.
Critique your approach, not just the employees
"I think the biggest challenge for folks using the kit sometimes is understanding the concept of social engineering and how you go about attacking an organization. You really have to understand how a company ticks in order to pull off a successful social engineering attack," said Kennedy.
So, when once you've completed your pen test, look back on what worked and what might not have to not only offer information that will help the organization shore up defenses--but also to see where you yourself may have come up short on researching your company.
"A lot of failures come as the result of pen testers who haven't done the research. The folks often just have a shock and awe mentality where they go in and just see if anything sticks. That almost always guarantees failure."
Sign up for Computerworld eNewsletters.