Newforma Project Center - Performance Characteristics

2/24/2009

Customers often don’t know what level of performance to expect when using Newforma Project Center. This can lead to wasted time looking for optimizations that aren’t possible. Even worse, customers might ignore slowness simply because they don’t know that a configuration change could make things much faster. What follows is a list of benchmark operations that we have performed at Newforma headquarters. We hope that this list will be helpful when used as a comparison for customers wondering if their system is optimally configured.

 

The performance of Newforma Project center is dependent upon a long list of variables. Not only is the Newforma application deployed on servers and desktop computers, each with their own performance characteristics, but it is also sensitive to other common desktop applications and enterprise anti-virus solutions. Network speed and latency can drastically affect many operations. In addition, project data sizes vary dramatically across our customer base, and larger amounts of data will often affect performance too. Because of these factors, it’s really hard to know if a Newforma deployment is performing optimally.

The Newforma team frequently gets asked about expected performance. Posting some examples of what our engineering team typically sees in-house seems like a good way to set some benchmarks for comparison. If your experience is significantly different from the cases below, we suggest contacting Newforma support - it’s possible they’ll be able to help configure your environment for improved performance.

 

Indexing Performance

Newforma indexing speed is primarily dependent upon the Newforma Project Center Server hardware and network connection speed to the file server hosting the project files. It is also highly dependent upon the content being indexed. In the test below, the NPC Server is a Dell 1950 two-processor (8 core) Xeon E5420 running at 2.50 GHz. It has two 15k 146 gb SAS drives configured as RAID 1. (This system scores 6966 on the Passmark CPU benchmark, see http://www.cpubenchmark.net/ for additional details.) We purchased this server for $2800 in November of 2008.

Newforma can disable our search Catalog Scheduling feature on NPC servers with 8 cores. This option allows all catalogs to index simultaneously, resulting in a higher search index throughput. On servers configured in this manner, indexing performance is typically limited by the server disk speed. In this case, a direct attached storage array with multiple spindles becomes beneficial.

In our performance tests, the content being indexed is actual project data collected from Newforma customers. It is a mix of files typically found in AEC firms that is broken into 57 projects, containing 1.1 million files, consuming 600 GB of disk space. The table below shows the time to create a 100% indexed search catalog in a variety of configurations.

1.1 Million files, 600 GB	Catalog Scheduling (default server configuration)	100 mbs network connection	6 days, 14 hours
1.1 Million files, 600 GB	Catalog Scheduling Disabled (an optimization possible for 8 core servers)	100 mbs network connection	2 days, 19 hours
1.1 Million files, 600 GB	Catalog Scheduling Disabled (an optimization possible for 8 core servers)	1000 mbs network connection	1 day, 21 hours

 

The amount of data an individual NPC server can index varies widely based on the content being indexed. It is dependent mostly on the “unique text density” of the files. We commonly see 1-2 tb of data in up to one million files indexed by a single NPC server. Some customers have as much as 5 tb of data indexed by a single server, although this number is above the typical limit.

 

File Transfers

Transferring a file from an NPC client to InfoExchange is usually pretty quick. Exceptions can occur if the InfoExchange computer is at a remote location. In this case, you will likely be limited by your location’s external network bandwidth. If your local area network transfers are slow, see the performance tips section below on how to create an alias to improve transfer speeds.

Transferring a file to or from InfoExchange when using a computer outside of your local intranet is typically limited by the external network bandwidth.

The transfer times below are for transferring a 20 mb file in a variety of settings

File Transfers:	Minutes:Seconds
Upload Local Area Network	0:22
Download Local Area Network	0:20
Upload home DSL:	22:10
Download home DSL	3:53
Upload home cable	2:30
Download home cable	0.39

 

Other Miscellaneous NPC Client Timing Tests

Opening projects from the local area network: 4-10 seconds

Opening a project at a distant office (WAN): 10-20 seconds

(The WAN times are for opening a moderately sized project located in California, from a client located in the New Hampshire office. Our New Hampshire office has a 5 mbs network connection, with 100 ms latency to the California server. The connection uses a Riverbed Steelhead network accelerator.)

Switching Activity Centers: Typically no more than a few seconds, even for very large projects

Opening an action item, RFI, or Submittal containing 200 emails: This performance has been significantly improved in Sixth Edition.

 

Searching for a text phrase: Searching a 20 gb project containing over 200,000 documents for a phrase matching 141 documents took 40 seconds.

Outlook: Outlook startup is affected when the Newforma plug-in is installed. This increase in startup time is largely due to loading the .Net programming framework. Newforma’s Outlook plug-in code does relatively little work during startup. (When the plug-in is initializing, you’ll see a Newforma progress bar.) The Newforma plug-in should not noticeably affect Outlook runtime performance.

Opening Outlook:	NPC Plug-in Enabled	Disabled
XP/2003	:08	:01
XP/2007	:10	:03
Vista/2003 64 bit laptop	:40	:16
Vista/2007 64 bit	:09	:01

 

Performance Tips

 

Indexing Performance

See the Newforma hardware requirements specification for information on our recommended NPC Server configurations. In general, fast CPUs with multiple cores can make a big difference in the indexing speed.

 

The disk where the Newforma search catalogs are being created should be a high performance disk, particularly if catalog scheduling is disabled. A 10k RPM SAS drive is a good minimum. IDE or Serial ATA devices will not keep pace with large search catalogs. Some level of RAID is a requirement.

 

The performance of the file servers that contain files being indexed will also have an effect on indexing service performance. Faster fileservers will improve the indexing performance. Slow NAS devices can be a significant liability. Indexing across a WAN is not supported, and can often lead to indexing database corruption.

 

A gigabit network connection between the file server and NPCS can significantly improve indexing performance.

 

 

Info Exchange address resolution

Access to Info Exchange from NPC and NPCS always has to occur using the common name provided on the SSL certificate (ex. infoexchange.newforma.com). If no special alias is setup this will resolve at a customer site to the externally facing IP address. This is not typically the quickest route to the server as many customers will have an alternate way to address servers in the DMZ. Below is a "tracert" to infoexchange.newforma.com from our internal network without any special alias. With an alias this might be reduced to a single hop and drastically improve upload/download performance for the NPC user.

Tracing route to infoexchange.newforma.com [67.217.105.21]
over a maximum of 30 hops:
 
 1    <1 ms    <1 ms    <1 ms  ...
 2     3 ms     5 ms     3 ms  ...
 3    10 ms    <1 ms     1 ms  ...
 4    16 ms     7 ms    6 ms  ...
 5   211 ms   338 ms    *     ...
 6    15 ms    12 ms    13 ms ...
 7    13 ms    12 ms    18 ms ...
 8    13 ms    14 ms    20 ms ...
 9    14 ms    13 ms    19 ms ...
 10   12 ms    12 ms    26 ms  ...
 11   33 ms    27 ms    20 ms  ...
 12   28 ms    22 ms    18 ms  ...
 13   37 ms    26 ms    27 ms  ...
 14   23 ms    22 ms    21 ms  ...
 15   24 ms    24 ms    25 ms  ...
 16   25 ms    27 ms    23 ms  ...
 17   27 ms    22 ms    22 ms  ...
 18   22 ms    22 ms    22 ms  ...

A couple of suggestions as to how to create the alias:
-   Put the direct address of the Info Exchange server into the /etc/hosts file on each NPC client. This can be done with a domain policy to push the file to the client so it does not have to be manually edited.
-   The domains DNS server could setup an alias infoexchange.newforma.com to point to the direct address
 
Anti-virus exclusions on NPC, NPCS and Info Exchange

Our deployment documentation talks about the areas of the servers to exclude from anti-virus scanning. Many AV programs will scan files on demand as they are accessed. This is a real detriment to performance and can cause data corruption in the worse case. Please consult the deployment documentation to ensure the correct exclusions are setup on the clients and servers.
 
Additional software on NPCS and Info Exchange

Our servers are developed and tested as dedicated servers. Our specifications call for a dedicated server for NPC. Still, some customers don’t follow our recommendations. While our software will be compatible with most other software, performance may suffer if the other software is using too many system resources. Using Task Manager and Performance Monitor is a good first place to check when a server is performing slower than expected. This will quickly identify any processes that are using resources. Our NPC servers have several processes that are part of our solution (N4, ProjectCenterServer, mysql, extractor processes). Info Exchange has only a couple (w3wp and mysql). Any significant use of resources by other processes is worth investigating.
 

Network and Remote (WAN) Projects

Opening projects across a WAN can result in a significant performance hit. Customer experiences vary widely in this area. Some customers, with good MPLS network connections between offices, report almost no difference in performance between WAN and local projects. Other customers may find the slowness affects them so much that working in this mode is not reasonable. Significant improvements in WAN performance are being worked on for the next release of NPC.

Virtual Servers

Virtual servers are markedly slower than physical servers. Newforma currently does not recommend deploying NPCS in a virtual environment. If your virtual machine is slow, consider deploying on non-virtual hardware.