The Great Debate: Automated vs. Manual Penetration Testing

The rules have changed. Today, regulators require most financial institutions to have regular network attack and penetration testing to ensure secure customer data. Do you know how to respond to the requirements and best assess your network security? In seeking proposals for attack and penetration testing, you will often find the scope of the testing to be broken down into one of two types: automated testing and manual testing.  These different approaches may or may not test the seven layers of security. The seven layers include: Perimeter; Authentication; Package; Configuration; Monitoring; Physical; and Policy Layers. Each scope has its advantages and disadvantages, but which method is truly the best? Let us take a closer look.

Automated Penetration Testing

When you hire a company to perform an automated scan, the company “installs” a device that collects data over time on your network. The device is placed on the network or it externally scans your devices from the Internet. Once turned “on,” the device collects as much network information/data as it possibly can in order to determine potential vulnerabilities. After a pre-established period of time, the company’s staff come back to collect the device, and a few weeks later, you get a report that shows network weaknesses that were discovered during that timeframe.  These scans, once initiated, are completely automated and do not require a human to perform their tasks. A technician must only install the device and then return after the set period of time to collect the device and data.  The technician requires minimal training, other than basic networking knowledge needed to install the device.

The advantages and disadvantages of automated penetration tests are listed below.

Advantages

  • Low cost (due to the lack of human interactions required)
  • Swift speed of reporting
  • Extended length of testing window
  • Ability to search for sensitive, pre-established data
  • Collection of large amounts of data from the network
  • Collection of data from entire subnets (sections) at one time

Disadvantages

  • Can test only four of the seven layers of security (Perimeter, Authentication, Package, and Configuration Layers)
  • Reports are often pre-generated and are created by untrained technicians
  • Do not perform pivot attacks (compromising one machine and then launching attacks from that machine to other areas of the network)
  • Often times do not verify exploits (eliminate false positives)

Manual Penetration Testing

When hiring a company to perform a manual scan, the company sends a trained tester to your site (or the tester tests your Internet presence externally) to run various scans and attacks against the network in an attempt to compromise customer data or breach important systems. Manual testing tends to focus on an approach of not relying on one single tool to collect network data/information.  It involves running multiple programs to gather and interpret the data and results in real-time, so the tester can adjust the attack plan as the engagement progresses. The advantages and disadvantages of manual penetration tests are listed below.

Advantages

  • Allow  pivot attacks
  • Can test all seven layers of network security including the Monitoring, Physical, and Policy Layers (not covered by automated scans)
  • In-depth approach, in terms of the detail, for each host reviewed
  • Allow intelligent searching of sensitive data
  • Put a trained “brain” behind the testing to interpret the data
  • Can eliminate false positives

Disadvantages

  • Longer length of testing
  • Often a more limited scope in terms of the number of systems tested
  • Usually more costly
  • Increased time in getting final deliverable

So, which method is better?  The answer comes down to what scope you are looking to cover.

Your Scope

From an external-only perspective, meaning that you want to know what Internet-based attacks can be successful against your network, hiring a trained professional is typically worth the extra investment. After all, trained hackers are going to perform their attacks slowly and manually in order to get the results they want.

From an internal perspective, the size of your institution, the number of network nodes (or systems) that your institution wants tested, and the amount of time you are willing to subject your internal networks to an attack all factor into the decision. One institution may determine that it is best to perform a manual scan against the critical devices on its network since the cost would not be that much greater than an automated scan. Another institution with several network subnets full of servers and workstations that wants to have the most thorough testing of all seven layers of security would most likely want to have an integrated approach where both automated scans and manual scans are performed. The manual scans should be limited in scope to devices that house important customer and corporate data. The automated scans could then search for workstation or network device vulnerabilities which would show susceptible hosts that may not house sensitive data but might be used for pivot-based attacks.  Ideally, the manual tester would have access to the results from a previously performed automated scan or be allowed to perform his/her own automated scans. From there, the tester can interpret the data and find the best means for attack.

The only precautionary note that should be made is that by relying solely on an automated scan, all seven security layers and other types of social engineering activities are not being tested. No computerized, automated scan can try to trick an employee into giving your passwords, customer information, or other corporate and personal data to unauthorized individuals. Nor can an automated scan access the physical security of your server room or determine what a monitoring system deployed at your institution is able to detect. Most penetration testers would agree that testing of these three areas can often show significant weaknesses in a well-maintained network because of one poorly trained staff member or one system that is accessible due to weak physical security.

Now do you know how to respond to regulator’s requirements and best assess your network security? With your customer data at stake, you need to be prepared for an attack.