Four strategies to prevent data encryption from hijacking your network

Phil Trainor, Head of Security business, Ixia, Asia Pacific | Oct. 26, 2017
Knowing that many organisations pass encrypted traffic into their networks without full inspection, the bad guys use encryption to hide malware and launch attacks, effectively hijacking your network.

This vendor-written piece has been edited by Executive Networks Media to eliminate product promotion, but readers should note it will likely favour the submitter's approach.

data encryption
Credit: Storyblocks 

While encryption can keep your network traffic safe from hackers and cybercriminals, it can also prevent your security and monitoring tools from seeing inside the packets crossing your network. Knowing that many organizations pass encrypted traffic into their networks without full inspection, the bad guys use encryption to hide malware and launch attacks, effectively hijacking your network. To keep defenses strong while limiting the risk of security breaches and data loss, you need to decrypt, examine, and re-encrypt all network traffic.

 

The burden of decryption

Devices for decryption must be powerful. Encryption algorithms are becoming longer and more complex to withstand hacking. A test done by NSS Labs several years ago found that moving from 1024- to 2048-bit ciphers caused an average performance drop of 81 percent on eight leading firewalls. However, SSL decryption does not need to be performed on a firewall. New strategies are available to offload decryption and send plain text to tools, enabling them to work efficiently and process more traffic. Here are four strategies to make decryption easier, faster, and cost-effective.

 

Strategy 1: Remove malicious traffic before decrypting

Many IP addresses used in cyberattacks are reused and known in the security community. Dedicated organizations track and verify known cyber threats on a daily basis, maintaining this information in an intelligence database. By comparing incoming and outgoing packets against this database, you can identify malicious traffic and block it from your network. Because the comparison is made with packet headers in plain text format, this strategy eliminates the need to decrypt the packets. Eliminating traffic associated with known attackers reduces the number of packets to decrypt. And, eliminating traffic that would otherwise generate a security alert helps security teams improve productivity.

The fastest way to deploy this strategy is to install a special-purpose hardware appliance called a threat intelligence gateway in front of a firewall. This appliance is designed for fast, high-volume blocking, including untrusted countries, and is updated continuously by an integrated threat intelligence feed. Once the gateway is installed, no further manual intervention is required, and no filters need to be created or maintained. Malicious traffic can be either dropped immediately or sent to a sandbox for further analysis. Depending on your industry and how often you are targeted, you could see up to an 80 percent reduction in security alerts.

Alternatively, you can configure custom filters on your firewall to block specified IP addresses. Unfortunately, firewall filters must be manually configured and maintained, and there is a limit to how many filters can be created. The explosion of connected devices and compromised IP addresses outstrips the capabilities of firewalls. Plus, using the processing cycles on an advanced device like a firewall to make simple comparisons is not a cost-efficient way to block traffic.

 

Strategy 2: Look for advanced decryption capabilities

Once the encrypted packets traveling from or to malicious sources is removed, a decryption device is needed to process the rest. Many security tools, such as next generation firewalls (NGFW) or intrusion prevention systems (IPS), include an SSL decryption feature. However, a paper issued by NSS Labs warned that some tools may not have the latest ciphers, may miss SSL communications that occur on non-standard ports, may be unable to decrypt at advertised throughput, and may even fast-path some connections without performing decryption at all.

Cryptography relies on advances to stay one step ahead of the bad guys. Security solutions need to support the latest encryption standards, have access to a wide variety of ciphers and algorithms, and have the power to decrypt traffic using the larger 2048- and 4096-bit keys as well as newer Elliptic Curve keys. As security technology grows in complexity, solutions must be able to process decryption efficiently and cost-effectively-without dropping packets, introducing errors, or failing to complete a full inspection.

As the volume of SSL traffic increases, the quality of a decryption solution is more important to achieving total network visibility.  In addition, Defense in Depth is a widely regarded best practice, which often involves multiple best-of-breed security devices (such as a separate firewall and IPS).  It is very inefficient for each of these devices to decrypt and re-encrypt traffic separately, which both increases latency and reduces policy effectiveness and end-to-end visibility. 

 

Strategy 3: Choose tools with operational simplicity

Another key feature is the ease with which administrators can create and manage policies related to decryption. This is important in industries that must comply with the mandates of Health Insurance Portability and Accountability Act (HIPAA), Federal Information Security Management Act (FISMA), Payment Card Industry Data Security Standard (PCI DSS), Sarbanes- Oxley Act (SOX), and other standards. The best solutions provide a drag-and-drop interface for creating filters and the ability to selectively forward or mask information based on pattern recognition (such as social security numbers). They also make it easy to keep a complete record of each SSL cipher used and all exceptions related to dropped sessions, SSL failures, invalid certifications, and sessions not decrypted for policy reasons. These detailed logs are valuable for audits, forensics, and network troubleshooting and capacity planning.

 

Strategy 4: Plan for cost-effective scalability

As the volume of encrypted traffic increases, decryption will have a greater impact on the performance of your security infrastructure. It pays to plan ahead. While it may seem logical to simply "turn on" the SSL decryption feature in a firewall or unified threat management (UTM) solution, decryption is a process-intensive function. As SSL traffic increases and more cycles are required for decryption, performance will begin to suffer, and tools may begin to drop packets.

To increase the flow of traffic through a multifunction device, the only option is to increase overall capacity. Adding capacity is a significant capital expense and some features have an extra cost to ensure the device can handle decryption.

A better option is to use a network visibility solution or network packet broker (NPB) with SSL decryption to offload security tools. Many organizations use NPBs to aggregate traffic from across the network, identify relevant packets, and distribute them at high speed to security tools. NPBs using hardware acceleration can process traffic at line rate with no packet loss, and can automatically load balance. They also eliminate the requirement for multiple inline devices to each perform independent decryption/re-encryption.  The cost of scaling an NPB is lower than scaling most security appliances, and can provide a quick return on investment.

 

Conclusion

As more of the Internet shifts toward encrypted traffic, attacks in SSL traffic will become more common. To protect data and networks from hackers and cybercriminals, it is essential to inspect all encrypted network traffic. An organization that does not develop a rigorous approach to inspecting encrypted traffic will undermine network security, creating an unacceptable risk of breach and data loss. Fortunately, new solutions are emerging that improve the efficiency and cost-effectiveness of SSL decryption.