Finding the right perimeter-based data loss prevention tool means striking a balance between speed, accuracy at detecting and blocking sensitive data from exiting the network, and adequate coverage across a broad range of rule-sets and protocols.
DLP products come in three categories: perimeter-based, client-based and those that take a combined approach. In this test, we evaluated perimeter-based appliances from Fidelis Security Systems, Palisade Systems, Code Green Networks and GTB Technologies.
DLP products come in three categories: perimeter-based, client-based and those that take a combined approach. In this test, we evaluated perimeter-based appliances from Fidelis Security Systems, Palisade Systems, Code Green Networks and GTB Technologies.
The DLPs were set up inline (except for Code Green's Content Inspector, which doesn't support in-line mode) between a simulated WAN and LAN and were configured with a set of 10 rules. We then ran about 1,100 files through each device, waiting about a minute between each file, to determine how accurately the device detected and blocked a total of 276 “bad” files and to what degree network performance was affected by the inline DLP.
Here are our key findings:
All of the products did an effective job detecting harmful files that were sent over the specific protocols that the product supports. But not all products support a wide range of protocols.
Some of the products that did well at detecting harmful files were less adept at blocking.
None of the products were able to analyze or block encrypted traffic.
There's a network performance hit that needs to be taken into account when running these products in-line.
Code Green's Content Inspector scored highest when it came to detection. Code Green also scored high on ease of configuration. But Code Green was limited in the range of protocols it could block.
Our Clear Choice winner is Fidelis' XPS because of its easy-to-use interface, flexible rule-set, amazing reporting, and better-than-average detection and blocking ability.
Palisade's Packetsure and GTB's Inspector were somewhat unrefined by comparison, requiring more work to understand the rule structure and adding unneeded complexity to the overall process. But they were still very competitive when it came to detecting harmful files.
Installation
Generally DLP vendors deploy engineers to the customer site to set up and configure the device, but we decided to do it ourselves to get a hands-on understanding of how the product works from installation through reporting.
For Packetsure and Content Inspector, the basic installation was fairly straightforward and the products were setup with little to no trouble. For the other two products, basic installation was a little more difficult, requiring numerous contacts – via e-mail and phone. But they eventually all were set up without the need for a technician to show up on-site.
After each product was set up and could pass data between the simulated LAN and WAN, we configured the device to our filtering specifications. This included a sample set of 10 rules chosen to test some of the basic features and blocking potential.
The DLPs were set up to look for Social Security and credit card numbers, certain pieces of source code, and five words in a row from a short story, which would be used to prevent any part of a specific report from leaving the network.
We also set up rules to check for maximum file sizes or .mp3 files. And we fingerprinted a data set containing a list of customer names, addresses and Social Security numbers and set up a rule blocking any combination of the three.
Configuration: Code Green is tops
Code Green's Content Inspector was the easiest product to configure and write rules for. The rule language is simple and the graphical interface is very usable. Code Green breaks rule creation down into two categories: data and policy. One defines data to be blocked using a variety of tools, and then configures a policy to check for it. This was very straightforward and easy to change, with no need to restart the device or reload the settings. In the configuration simplicity arena, Code Green goes above and beyond all the other products.
Fidelis' XPS sensor has a “Command Post” server to handle management and configuration, a mail sensor server (provided via built-in Postfix SMTP proxy), and a Web sensor (implemented via a third-party BlueCoat Web proxy appliance).
Rule creation is straightforward and simple using a Web GUI. XPS is the only product that allows you to submit sample files in order to test each rule before you make it live.
If you ever have a question about a specific rule or a page you are on, Fidelis has built in wonderful help links on each page that explain each check box or button. This is a life-saver and allowed us to create the majority of the rules without any technical support contacts.
Palisade's Packetsure provided a simple wizard to help with setup and was the only product to have such a helpful starting point. However, if one wants to add or change a rule outside of the wizard, the sailing is not quite so smooth.
Part of the problem may be that Packetsure is really two products trying to work together as one: there is a content analysis engine and a protocol analysis engine. The Palisade protocol analyzer only inspects the packet payload (instead of reassembling the data stream as the content analysis does). This two-pronged approach helps isolate each rule, but it makes managing the product difficult.
Also, in our testing the rules did not always work as expected. For example, one “content analysis checkbox” means packet analysis and another content analysis checkbox actually reassembles the data stream before it analyzes it (similar to all the other products).
Packetsure has a “connect to home” functionality which the user can enable right out of the box. This feature can be very useful when calling tech support or even with the initial setup as it allows Palisade to assist using a secure VPN.
GTB's Inspector has the most difficult configuration process of the four products. In order to write a rule, one must edit a text configuration file, add some regular expressions and format each line very specifically. For example, in order to write a rule to check for the words “Top Secret” in a file, a regular expression had to be written in a large text box on the Web management interface.
There is no wizard and no graphical interface. The other limiting factor with GTB's Inspector is the fact that its rule-set functionality is very limited. In our test it could only implement about half of the desired rules. Even a simple rule such as looking for specific filenames or maximum file size was not supported.
Performance: Fidelis is fastest; Code Green wins detection test
We tested how accurately the product blocked a total of 276 harmful files that we sent, or roughly 30 files for each of the nine protocols (including HTTP, SMTP, POP, IMAP, FTP and Telnet) in our test bed. We also measured how fast the product could pass data through the device, starting with a baseline of 581Mbps, which is the capacity of our network without any device present.
The best performance from a detection perspective was Code Green's Content Inspector, which detected 90% of the data we threw at it. And the 10% Content Inspector missed was because of the lack of support for encrypted traffic streams (SSH sessions), which no product supports.
However it can only block files on four of the tested protocols: HTTP, Secure-HTTP, FTP and SMTP. The first three are done using a third-party BlueCoat Proxy device and the SMTP is done using a built-in mail relay.
This lack of blocking ability across a wide variety of protocols was the major drawback in Code Green's Content Inspector. But if your company is only worried about those four protocols, this product would be recommended.
Fidelis' XPS had an 84% success rate in detecting and blocking across all protocols and streams of data. The marketing line for this company states that they can block data on all 65,535 ports and we would have to agree. This product blocked virtually everything it could detect, only failing on one file type – an archived Web site.
The product handled obfuscated data very well — catching four of five files. POP and IMAP provided a little bit of trouble, but after a few custom patches from the engineers, it worked as expected.
The choice faced by all these products is a tradeoff between performance and blocking effectiveness. When data moves through a DLP device, the product can choose to either cache it, determine that it's good and then let it out, or try to do analysis on the fly, and suffer some data leakage.
Fidelis chose performance and won our speed test, passing traffic at 90% of network capacity. However, occasionally pieces of sensitive data leaked from the network. All the other products chose to prioritize blocking over speed.
Palisade's Packetsure is targeted at the basic protocols of HTTP, SMTP and FTP, and showed a high blocking rate on those specific protocols. But Packetsure, possibly because it seems to contain two products in one, was the slowest product, performing at only 55% of the allowable bandwidth.
Furthermore, blocking a specific protocol and scanning based on content analysis work as expected, but when you combine the two, problems emerge, creating unexpected results. For example when you try to limit content analysis to a certain protocol, you have to choose between using a weaker content analysis system (which won't reassemble the stream) or not limit your blocking based on protocols. The latter is the best way to handle this problem, but doing so reduces the flexibility and blocking capability of the product.
GTB's Inspector was the most consistent product. What it detected and blocked on one protocol it detected and blocked on every protocol with no extra work. The problem with this product was it only could check based on certain rules and those rules were limited. About half of our detection tests failed on this product because the rule types are not supported. However, even with its lack of rule support, it still caught 62% of the illegal files.
Across supported protocols, Inspector was the only product to score a 100%, catching every single file we could send through the machine at 80% of the allowed bandwidth.
Fingerprinting: GTB Inspector gets high marks
Fingerprinting is a concept that is implemented fairly well in these DLP products. Fingerprinting will hash a file and look for parts of that file leaving the network.
Fingerprinting is used to prevent sensitive information from leaving a network and at the same time to reduce false positives. For example, most organizations want to prevent Social Security numbers from leaving local networks. However, a lot of things can look like a Social Security number (e.g., a mistyped phone number or an online order number).
Fingerprinting takes any sensitive information you may have on your network and looks for a number of pieces that specifically correspond with it, to make it a piece of information that you don't want erroneously leaving your network.
One could fingerprint a list of names, addresses and Social Security numbers and, instead of triggering on any nine-digit number, the DLP will only trigger when a Social Security number is sent out with the associated full name. Or, instead of looking for a specific word phrase, it can look for a few sentences from a report.
All of the tested products support this feature, but GTB Inspector is the most powerful and flexible — customers can fingerprint data from a variety of flat files, databases or spreadsheets.
That power and flexibility, however, comes at the cost of simplicity. GTB has its own program which one must use to fingerprint data, as opposed to other products that allow an administrator to upload and fingerprint a file from the main management interface.
While Palisade’s Packetsure can scan and hash the usual range of files that most of the others support, when it comes to database fingerprinting – linking two fields in a relational database – it requires the files to be exported into a flat file for analysis. Fidelis' XPS included the ability to test your fingerprints once you created them.
