Virtualization security remains a work in progress

While adoption of server virtualization is proceeding at a gallop, the effort to refine virtualization security reached only a slow trot in 2009.

Roughly 18% of server workloads have been virtualized, and research firm Gartner expects that number to climb to 28% in 2010 and almost 50% by 2012. But adapting traditional firewall, intrusion detection, antimalware and other types of security and monitoring software to run optimally in this radically changed hypervisor-based architecture is still very much a work in progress.

One development that occurred this year is the release of VMware's security APIs.

After talking up the idea since February 2008, VMware in April 2009 finally released its VMsafe APIs intended to help security vendors build products to work with its platform. But some vendors say these APIs present performance issues.

“We're not using the VMware APIs today due to performance,” says Richard Park, senior product manager at Sourcefire, which in early December shipped its first virtualized sensor and management console for VMware ESX and vSphere4.

Sourcefire's traditional physical appliances are network sensors that can do both intrusion-detection monitoring and intrusion-prevention blocking. But at this point, the Virtual 3D Sensor and Virtual Defense Center will only provide monitoring visibility into VMware's ESX hosts, not blocking of attacks.

“The only way to block traffic today is to put the sensor between two VMware switches,” Park says. Sourcefire is still examining exactly how to fully support that. For customers today with VMware-based virtualized servers, “the demand is for monitoring,” Park claims.

Park says Sourcefire is eager to see a robust set of VMware VMsafe APIs and that VMware has recognized there are performance issues and is revising its APIs.

At the Gartner ITExpo in October, Gartner Vice President Neil MacDonald publicly excoriated some security vendors for not moving more rapidly to come up with software-based virtual appliances, insinuating they would rather stick to their old ways of selling expensive hardware boxes. (See related story, Gartner on cloud security: “Our nightmare scenario is here now”.)

Enterprise customers are rapidly virtualizing their IT environments and often unwittingly creating less-secure results even as they reap the many benefits of virtualization, MacDonald says. Roping off virtualized servers with virtual LANs alone — a common practice — “is not sufficient for security separation,” MacDonald says. “It's become the default because it's built into VMware with its virtual switch. Our position is it isn't strong enough.”

MacDonald says virtualization is causing some “business-model disruption” in security and praised the efforts of some vendors, including Trend Micro, to leap in with new offerings to take on the virtualization challenge. Using the VMware VMSafe APIs is one approach which is still new, he noted.

Trend Micro's Core Protection for Virtual Machines, antimalware software that was designed for use with VMware, was released in the third quarter. Trend's Deep Security 7 for firewall, intrusion detection/prevention, integrity monitoring and log management for VMware ESX shipped in November.

According to Bill McGee, senior director of product marketing at Trend Micro, both products make some use of tools in VMsafe. But he adds that while VMsafe is an important step, it needs to be improved.

“VMware is making improvements in the area of performance for bandwidth and significant workloads,” McGee says, especially by changing the approach they use for “sending packets around in the system.”

Virtualization is bringing change and “we're seeing the pressure, and the opportunity, for security vendors to optimize security,” McGee says. VMware has been among the most aggressive of the virtualization software vendors to open up their technology to optimize security functions, he says, while so far the actions of Citrix and Microsoft seem “more limited” in this area.

For its part, VMware says it's glad to see a number of vendors, including Altor Networks, Reflex, ISS IBM and Trend Micro, adopting the VMsafe technology.

While not speaking to specific comments about performance, VMware's director of alliances Jitesh Chanchani says, “VMsafe is an integral part of our security strategy. In terms of improvements, this is an ongoing investment for us.”

The APIs are a positive development, he points out, because they “provide fine-grained visibility into virtual-machine resources,” such as the introspection ability to examine what's going on the VMware platform.

Meanwhile, industry watchers continue to address the question of whether adopting a virtualization platform brings more risk.

According to Forrester Research, adding hypervisor technology (Citrix Xen, VMware vSphere and Microsoft Hyper-V) “does add some marginal risk to IT environments, because it layers additional software on top of existing operating systems. All software, no matter how thin, contains hidden design mistakes and inadvertent coding flaws.”

Mistakes are going to be made and there will be attacks against virtual servers, the firm states in a report titled “Fear of a Hijacked Planet.” These can include an attacker who successfully compromises a virtual machine going after hosts, subversion of hypervisors, and live migration impersonation.

“On the user side, enterprises are collectively a bit confused. IT security staffs, in particular, have more questions than answers,” says Forrester analyst Andrew Jacquith. IT teams are asking questions such as “Is the hypervisor secure? Is the IT ops team doing something they shouldn't? What visibility do we have to the virtual machines?”

According to Jacquith, one disappointment remains VMware's Live Migration feature for configuring VMs so that they automatically migrate from one farm host to another, for purposes of fault tolerance and business continuity. “That's all good, except that the VM itself moves over the network in the clear, which makes a man-in-the-middle attack possible,” Jacquith notes. But he's optimistic improvements are coming in that arena, too.

On the plus side, Jacquith points out, the VMsafe program, along with more options from vendors for offline patching and update capabilities, means there's been progress in security virtualization this year.