Multiple regulatory compliance requirements have made changes as of late that involve adding Penetration Testing to be performed on an organizations environment. With this added requirement, comes added complexity and typically added confusion to the project. Most the confusion is a direct-result of organizations being uninformed on the differences between penetration testing and vulnerability scanning/assessments.
For the record, a vulnerability scan (or vulnerability assessment) looks for known vulnerabilities in a system and reports potential exposures. A penetration test is designed to actually exploit weaknesses in the system architecture or computing environment. There is a significant importance in knowing the difference between these two exercises prior to engaging with a security vendor, and this knowledge can be beneficial in limiting any surprises that may surface as a result.
Many companies market and advertise capabilities in penetration testing, but potential customers of these vendors would be smart in researching their methodologies prior to entering into any agreements. For example, the Payment Card Industry Security Standards Council (PCI SSC) now requires as part of PCI DSS version 3.0 that penetration testing in relation to PCI compliance requires the penetration testers follow an “industry-proven methodology” such as NIST SP 800-115 or Open Web Application Security Project (OWASP). This can only be in reaction to the market being flooded with vendors marketing penetration testing services and yet, only providing a “glorified” vulnerability assessment.
Identifying Penetration Testing and Vulnerability Scanning
Aside from the definitions provided in our opening paragraph, there are major differences that should be identified and agreed upon between any organization and potential security vendor.
- Period of Performance: As a general rule of thumb, vulnerability scanning should be a continual exercise and conducted at least quarterly and/or as new equipment is introduced into the environment. Conversely, penetration testing should be conducted less frequently (once per year) regardless of changes in the environment.
- Reporting Capability/Results: Penetration testing reports are generally short explanations of what information was captured during the testing, while vulnerability reports are detailed baselines of vulnerabilities. In addition, vulnerability assessment reports should be collected as they occur to ensure gaps are identified and mitigated.
- Compliance Requirements: Both are required by PCI DSS, GLBA, and FFIEC regulations.
- Work Requirements: Typically a vulnerability assessment can be performed by in-house staff or outside vendors, while “true” penetration testing should be provided by a third-party.
- Overall Purpose: Probably the most commonly confused, and arguably the most important aspect of the two is its purpose or intent. Penetration tests are utilized to demonstrate various exploits and as a tool to reduce exposure. Vulnerability scanning is used for detection purposes and demonstrates when devices could be compromised.
Conclusions Penetration Testing, Vulnerability Assessment, Compliance
It’s simple to see that both a vulnerability assessment and penetration test could be used to improve the overall security posture of an organization, but identifying the differences is important in setting result expectations. Asking simple questions about the process/methodology utilized, and the expected results of reporting can have a profound impact on your selection of a proven security vendor.