April 2, 2008 -

Server virtualization is a promising technology that is interesting to nearly every Newforma customer.  Virtualization promises to lower cost, simplify administration, reduce space and power requirements, and simplify load balancing and backup.  However, many Newforma customers have been unsatisfied with a virtual deployment of a Newforma Project Center Server.  This document provides background on Newforma virtualization efforts and discusses the circumstances where a production deployment of a virtual Newforma Project Center Server may be possible.  Note:  because of numerous problems associated with virtual deployments in production use, Newforma does not recommend virtualization of the Newforma Project Center Server at this time.

Virtualization is a storm sweeping the landscape of information technology.  Rarely has a new technology been so eagerly accepted in IT.  Gartner estimates that over a half-million virtual machines are currently in use.  By 2009 that number is estimated to grow to over three million machines.  (Estimates are from http://www.cio.com/article/108403/Gartner_Virtualization_Is_Too_Expensive.) The press and industry ‘buzz’ is so uniformly positive on the benefits of virtualization, that few people are aware of limitations with the technology.
Virtualization is most successful when used to combine servers or applications running on hardware that isn’t fully utilized.  By combining many different under-utilized physical servers into a set of virtual machines running on one physical device, IT departments can often gain significant cost savings by getting more efficient use of computer hardware. 
Virtualization offers fewer advantages when used to combine fully-loaded servers, or disk intensive and network I/O intensive applications.  This is because all the virtual machines still have to share one set of physical components: i.e.  CPU, memory, disk, and network.  The CPU and memory hardware can be scaled by purchasing multi-core servers with lots of RAM.  It’s harder to scale out disk and network bandwidth, although solutions such as multiple NICs and fast disk arrays connected by fiber channel can make significant improvements.  The disk or network latency imposed by the overhead of a virtual environment can significantly affect application performance.  In many applications this overhead is insignificant.  However, there are applications that don’t work well in a virtual environment.  For example, database virtualization can work well, but only when most queries are resolved from an in-memory cache (see http://www.vmware.com/technology/virtual-infrastructure-apps/oracle.html)
The Newforma Project Center Server is an example of a resource intensive server that relies on very fast low-latency connections to the disk and network.  This is the most difficult type of application to virtualize.  Our experience has shown that the virtualization overhead is so large that it prohibits production use in many production environments.  In addition, customers have found that virtual environments hosting Newforma Project Center Server can fail, requiring a hard restart of the physical server.  This type of failure can easily corrupt the search catalogs maintained by Newforma Project Center.  In this case, a rebuild of the search database is required – a step that can take many days.
To address the problem of virtualization overhead, vendors suggest purchasing high-end servers with very fast network and disk systems.  In addition, experts agree that a native virtual environment (a bare metal install) is preferable to a hosted virtual environment (a virtual machine that runs on top of an operating system).  The two most commonly used virtual environments are the VmWare ESX Server, which is an example of a native virtual environment and Microsoft Virtual Server 2005, which is an example of a hosted virtual environment.
Today, the best virtual environment for Newforma Project Center server would consist of VmWare ESX running a virtual copy of Windows Server 2003.  This would be deployed on a dual quad-core processor with enough RAM to allocate 4 Gig to the Newforma virtual server.  The system would reside on a directly attached storage array with a fiber optic connection to 15k rpm SAS disks.  It would also have multiple high-speed NICs.  Note that a system configured like this is significantly more expensive than a dedicated NPCS server! 
Case Stories
There have been many customers that have tried deploying Newforma Project Center in a virtual environment.  Many have started with the free virtual environments available from either VMWare or Microsoft , and have run them on existing server hardware that often doesn’t meet the Newforma standard hardware specification.  These types of deployments are doomed from the start.  They invariably lead to frustration at the customer site and a rapid change to a real physical server.  Rather than focus on these numerous failures, the following case stories present the more successful virtual deployments that we know of today.
Today’s success stories all share the following:
1. VmWare ESX Server
2. High-end computer hardware
3. A relatively low number of Newforma projects
4. A relatively small amount of project data to index

A large engineering firm:  One of the earliest virtual deployments on high-end hardware occurred at a large engineering firm that has been a long time Newforma customer.  This firm purchased an 8 core server with 16 gb of RAM and a RAID 5 disk array with 15k rpm SAS drives and used the VmWare ESX Server native virtual environment.  They successfully ran a single copy of the Newforma Project Center Server in this virtual environment for several months.  It was used to index about one terabyte of project data.  Flush with success of running one server, they added a couple of additional virtual servers.  Performance suffered and the Newforma Project Center server was unable to keep the search index current.  Even worse, “lock ups” of the virtual environment occasionally required a restart of the physical server.  This type of failure sometimes corrupted the Newforma search index database, requiring a rebuild of the database that took days.  After months of trials, the customer replaced the Newforma virtual server with a dedicated blade server and there have been no further performance issues.
An architecture firm with a small number of projects:  One of our customers deployed Newforma in a virtual environment in late 2006 using the fastest hardware they could buy at the time.  It consists of a HP DL360 G5 computer containing two dual-core 3 Ghz Xeon processors and 16 gb of RAM.  The server is connected to a SAN using two gigabit Ethernet connections and runs the VmWare ESX server.
This hardware runs three virtual servers:
(1) CAD server licensing and installs
(2) Main file server
(3) Newforma Project Center Server

They have allocated only one core and 4 gb of RAM to the NPC server.  (Note this is lower than the Newforma recommended hardware specification.) The company has 75 users, typically half use NPC each day.  They have only 30 projects, but the project data that they index is nearly 400 gb.  They’ve been running NPC in a virtual environment with no issues for over a year.
An architecture firm just starting NPC use:  Another successful virtual production deployment is at an architecture firm with 10-15 users and less than 40 projects.  Their hardware consists of five identically configured Dell 1950s.  Each has a dual quad-core 2.66 gHz processor with 32 gig of RAM.  In addition, each server has 4 network connections, and is connected to a SAN via fiber channel.  They are using the VMWare ESX Server.
The company has five of these computers, three are configured as ESX virtual boxes in a clustered failover environment.  Two are dedicated to their production SQL server and their Exchange server.
The three virtual boxes run 36 different servers. The virtual box that is currently running NPCS is also running 4 Deltek servers, 2 application servers, print servers, SQL reporting, SQL development, Blackberry Enterprise, and a GIS application server.
The IT team noted that their virtual performance markedly improved when they started running virtual machines that were created on the SAN, rather than on the local server disk. 
When fully implemented, this company expects to have 500-750 projects with about 500-600 gig of project data accessed by 150 users.  Our past experience has shown that this level of use may cause performance problems in a virtual environment.  We’ve asked the customer to closely watch performance as their NPC use grows.

Virtualization Options that work with Newforma
Newforma Info Exchange is a server that works well in a virtual environment.  Several customers are running successfully in this mode.
A Newforma Project Center Server running on a physical server may have CPU and memory cycles that could be used for other tasks.  One possible way to leverage these unused compute cycles would be to run hosted virtual machines on the server.  In this mode, virtual machines exist on top of the Newforma Project Center Server operating system.  These virtual machines will compete for system resources, but the overhead of running inside the virtual environment is avoided for the Newforma services. Virtual environments with non-intensive needs such as print, license, or applications servers are particularly good candidates for this type of environment.

Virtualization research and development at Newforma
The Newforma Engineering group is constantly using virtual servers for development and testing.  As hardware and virtual environments improve we will continue our scalability testing and hope to be able to provide improved support for production use of Newforma Project Center Server in a virtual environment in the future.