Virtualization a challenge for IP contact centers

IP contact centers can benefit from virtualization as much as any other technology, but because of the real-time nature and need for reliability, deploying virtual machines in this environment requires care in selection of hardware, segmentation of applications and the use of replicating virtual machines on the fly.

Contact-center elements such as voice, interactive voice response (IVR), automated call distribution (ACD) and conference bridging require near-real-time response, so virtual machines supporting them need enough dedicated horsepower, says Patrick Conroy, CTO of contact-center software vendor Callfinity, which also installs its products.

Designing the contact center requires dedicating CPU and memory to the real-time applications that need them, Conroy says. Virtual machines not running real-time applications can share remaining resources without dire consequences.

Contact centers also include less-time-sensitive business applications that pop up on call agents' PC screens to handle callers' needs and that can handle greater delay.

“So your file server or your Exchange server or whatever the case may be may wind up running low on CPU or RAM, but it's not as critical as your phone system or your ACD or your IVR,” he says.

Element separation

Separation of contact-center elements on different servers also is important, Conroy says. For instance, databases tapped by multiple applications should be put on separate servers where they are unaffected by CPU and memory demands of other applications to ensure their availability.

In a typical deployment, Conroy will place a single, critical contact-center application such as telephony on a physical host with other applications that are less critical, “so if you need more capacity for your telephony layer, you take away resources from your less important virtual servers.”

Others take a more cautious approach. For instance, contact-center vendor Aspect Software deploys only its own applications on virtual servers within a given physical host, says Roger Sumner, senior vice president of technology and architecture for the firm. No third-party business applications are allowed, he says.

“Over time I think the technology will be there within virtual machines to allow mixing applications,” he says, “but we'll be very guarded in that area because we want to ensure that applications get delivered in an appropriate amount of time.”

Meanwhile, Aspect is gathering data about the effects that other applications on the same physical host have on the real-time data center applications. Virtual environments allow setting of parameters that help ensure contact-center applications get the performance required from CPU, memory and bandwidth, says Sumner. “But when you exceed those parameters the behavior of our applications isn't as predictable,” he says.

The goal is to set benchmarks that help customers design virtual deployments that mix call-center and business applications on the same hosts and allow for live migration without risking such disturbances.

“We want to show them those contact-center applications that mix well with business applications,” Sumner says, so customers maximize use of server resources and have flexibility about where applications can reside.

On the fly

Virtual environments can create new virtual machines on the fly to address performance dips, but this live migration can cause separate problems, Sumner says. “There can be as much as a five-second disruption. If you're switching voice, that's fairly significant for us to deal with,” he says. So again he steers clear of live migration for time-sensitive applications.

In any case, live migration characteristics vary from vendor to vendor of virtualization platforms, and businesses need to know about the differences. “Be aware of what their live migration characteristics are and design around them,” says Chris Wolf, an analyst with the Burton Group. “That might result in more VMs that average a smaller load versus, say, fewer VMs that run a higher concurrent load.”

Live migration can result in proliferation of virtual machines, which in turn creates more challenges, says Francois Richard, the director of infrastructure engineering at Nuance Software, which makes speech-recognition software used in contact centers. Monitoring should be segmented so like applications can be viewed at once. For example, if speech-recognition software is located on multiple virtual machines on multiple physical hosts, monitoring should be set up to view all those instances at once, he says.

Some contact-center delays can be pinned on the hardware used to support the virtual machines rather than on the software, says Wolf. Converting virtual-machine demands for CPU, memory or I/O capacity creates an overhead as these virtual demands are translated into the use of physical resources. These translations take time that can cause delays in applications that require frequent updates to these page tables, Wolf says.

AMD has developed chips to handle these translations more quickly in hardware, and Intel is close behind, he says. So businesses should consider these chips when choosing hardware on which to run contact-center virtual machines.

“In the past I've seen organizations blame latency issues on network or storage I/O when in fact it's directly related to physical memory latency,” Wolf says. “Businesses have been really afraid of the latency issues and of being able to do virtualization on a large scale, but with this hardware I see more of them looking to go in that direction in 2009.”

High availability — redundancy that is critical to contact center reliability — is another challenge in virtual environments, Wolf says. “Your typical high-availability solution today is only able to detect failure of the physical server and relocate the virtual machine as a result of that,” he says. “In general it has very little insight into what is happening inside the virtual machine.”

The virtual machine as black box

The virtual machine is treated as a black box, he says, which means an application can hang up inside a virtual machine without the high-availability software knowing about it. “That virtual machine would continue to operate as if nothing is going wrong,” he says.

Similarly, if server hardware fails, the virtual environment needs to respond quickly to maintain calls, Wolf says. “What I mean by that is if there is a partial failure on the physical host I don't want complete loss. I might have, say, a network card go down, which in a contact center can be critical in terms of the amount of I/O I'm going to lose,” he says.

In that case it would be desirable to live migrate the virtual machines on the affected host to a healthy machine. One solution called continuous availability keeps a hot standby virtual machine ready in the same state as the active virtual machine.

“With that I'm able to run a single VM on multiple physical nodes simultaneously and keep that virtual machine in lockstep so if I do have a physical node failure, the VM continues to run, and I do not lose any application state as a result,” he says. Marathon Technologies has such a product, he says. “That's the type of intelligence that still needs to evolve,” he says.