Gaming the system with "independent research"

Oracle's PR firm just send me a message touting a new study presented by the Evaluator Group that showed that it is easier and quicker to install Oracle VM in an environment supporting Oracle RAC and Oracle Business Suite than installing VMware's vSphere. The main supporting reason was that Oracle's templates for those products drastically reduced the time of installation. As any teenager would say, "Well, duh!"

Here's what Oracle's press release has to say

Leading Analyst Firm Finds Oracle VM Beats VMware vSphere in Time to Deployment for Key Oracle Applications

Users Can Deploy Oracle Real Application Clusters Up to 10 Times Faster with Oracle VM Templates

Redwood Shores, Calif. – August 24, 2012

News Facts

In a recent report, Evaluator Group found that deploying Oracle Real Application Clusters and the Oracle E-Business Suite using the Oracle VM Templates on Oracle VM was much faster than a traditional install performed with VMware vSphere 5.

Using Oracle VM Templates, the Oracle Real Application Clusters 11g Release 2 configuration deployed in just under 4 hours. With VMware vSphere 5, the same configuration took nearly 40 hours to deploy. In a similar comparison, the Oracle E-Business Suite 12.1.1 configuration deployed in two hours and 15 minutes, vs more than 15 hours. Requiring nearly zero application or operating system installation knowledge, Oracle VM Templates help substantially reduce the risk and time needed to deploy enterprise applications. In non-Oracle VM environments, customers can spend many months building similar solutions and fine tuning them, which typically requires in depth product knowledge.

Providing Value Beyond Server Consolidation

Oracle VM Templates enable the deployment of a complete application runtime environment within a virtual machine. The operating system, database and applications are installed, updated to the correct patch levels and automatically configured at the initial boot time of the VM, without any additional system or application installation tasks required. Certified by Oracle, the templates allow IT organizations to simplify and speed virtualization and deployment of their enterprise applications within traditional or cloud-based infrastructures and reduce deployment time down to hours and minutes rather than days and weeks.

Oracle offers more than 100 Oracle VM Templates, available for Oracle Database, Oracle Real Application Clusters, the Oracle E-Business Suite, Oracle’s PeopleSoft, Oracle Linux, Oracle Solaris, Oracle Fusion Middleware and many more. Customers can also create their own Oracle VM Templates for non-Oracle applications.

Oracle VM is optimized to accelerate deployment and simplify management of the entire application stacks. With integrated management, through Oracle Enterprise Manager, testing, and support from applications to disk, Oracle’s unique approach to virtualization not only helps consolidate IT resources — it enables IT to deliver cloud services rapidly and efficiently.

Let's pick this announcement apart, shall we?

Here are my thoughts after having read the press release and then reading the Evaluator Group report:

Vertically integrated products versus the real world.

Oracle has done quite a bit of work to make its Oracle VM easy to use in a Oracle-centric environment. It would be expected that this integration work would speed the installation process.

Does this mean that Oracle VM is a better virtual machine software platform than others? Well, no, it doesn't. It just means that a tightly integrated, read captive, system installation is easier than using a general purpose product.

Is this really a neutral, independent study?

Since a real world control  that is an environment containing a mix of Oracle and third party products, wasn't shown in the study, I was moved to ask the Oracle PR folks to find out who sponsored this study that Oracle was touting.

It turns out that Oracle sponsored this study.

This information isn't in the press release nor was I able to find it in the report.

Do you suppose this is a bit misleading? I do.

Would Oracle VM fare as well in a real world test?

A real world test would compare Oracle VM to VMware vSphere in an environment that is more like what is found in data centers around the world. You know what I mean. Data centers with a mix of operating systems, applications, development tools, database engines and management environments.

While it would have been nice if the "independent study" looked at real world environments, this type of test wasn't presented in the study results.

Snapshot analysis

It is highly suspicious that this study was released immediately before VMware's VMworld bash. It is clear that Oracle wanted to rain on VMware's parade and wanted to win so badly that a few rules of independent research were "bent" to make Oracle's virtual machine product look better than the industry leader.

All I have to say is shame on Oracle for doing this. Shame on the Evaluator Group for going along with a comparison of multiple vendor's products sponsored by only one of the vendors.

When I was at IDC and later at the 451 Group, this sort of study was not allowed by company policy.

Kusnetzky Group LLC © 2006-2014