Univa Acquires Grid Engine Expertise

Addition of Founding Sun Grid Engine Team Advances Data Center Optimization Strategy

Lisle, IL, January 18, 2010 – Univa, the data center optimization company, today announced that the principal engineers from the Sun/Oracle Grid Engine team, including Grid Engine founder and original project owner Fritz Ferstl, are joining Univa. The company will immediately offer superior engineering support for Grid Engine versions already installed and will publish a Univa version of Grid Engine before the end of Q1 2011. Univa will concentrate on improving Grid Engine for technical computing and HPC use cases in addition to promoting the continuity of the Grid Engine open source community.

As part of this announcement, Mr. Ferstl has been appointed Chief Technology Officer and will direct Univa's technology strategy as well as lead the company's growing EMEA business.

Grid Engine, a policy-driven workload and resource management system that automates workload distribution across a network of servers, is the most widely used system of its kind with over 4 million CPUs under management. The addition of Grid Engine technology and expertise directly supports Univa's data center optimization strategy by bolstering its workload and resource optimization capabilities, which already include the leading desktop management product, Grid MP.

"The first step to data center optimization is effective sharing of systems and workload distribution, and now Univa provides the leading resource management system as a primary element in our optimization stack," said Gary Tyreman, CEO of Univa. "Grid Engine is an area in which Univa has invested significant time and energy over the past 3 years, and we believe we are now best positioned to lead the pursuit of top HPC use cases and requirements that serve our most important customer sets. With this move, Grid Engine users gain a predictable and sustained future for Grid Engine that might not have been available otherwise."

"Combining the Grid Engine and Univa technology offerings was a once-in-a-lifetime opportunity that the new Univa EMEA team and I just couldn't miss," said Ferstl. "Now we'll be able to interact with and serve users worldwide investigating and understanding their data center optimization needs and providing the most innovative and enterprise-strength solutions to meet those needs."

Univa now offers the broadest and most innovative set of product capabilities for optimizing data center efficiency. Univa software continuously and dynamically optimizes an organization's efficiency with respect to workload and resources, people and processes, and application and license usage. This unique approach helps organizations earn rapid ROI through faster time-to-results and by keeping larger costs under control.

"Univa can now claim the most comprehensive set of category-leading products," added Tyreman. "We offer Grid Engine, the leading workload management solution; Grid MP, the leading desktop solution; and UniCloud, the leading dynamic cloud management software stack."

"I'm very excited by Univa providing continuity and ongoing support for Grid Engine," said Wolfgang Gentzsch, former CEO and President of Genias Software which originally developed the Grid Engine software acquired by Sun. "This is clearly a win-win-win situation: GE's thousands of users now have access to enterprise-class support at Univa, Univa customers benefit from the extended capabilities now available for fully optimizing data center efficiency; and the GE open source community can rest assured that robust development will continue thanks to Univa's commitment to the project."

"Univa developing and supporting Grid Engine is welcome news for the GE community," said Ian Foster, Professor of Computer Science and Director of the Computation Institute at the University of Chicago and Argonne National Laboratory. "This move by Univa brings with it a new competitive spirit and sense of innovation to a market that continuously benefits from pushing the envelope."

"This move propels Univa into a lead position in both the data center optimization and workload management spaces," said Steve Tuecke, co-founder of Univa and Deputy Director of the Computation Institute at University of Chicago and Argonne National Laboratory. "We selected Grid Engine as a core part of Univa's software stack almost five years ago due to its market leading feature set, large user community and vibrant open source community, all backed by a fantastic engineering team. Now that key members of the Grid Engine team are part of Univa, it seems a no-brainer for Grid Engine users to go with Univa for ongoing GE technology and services, as well as the broader data center optimization stack Univa has built around GE."

For additional quotes/reactions from the user and technology community, visit: http://www.univa.com/about/quotes.php.

Grid Engine support is available immediately from Univa, and the Univa version of the product will be available by end of Q1 2011. For more information about Univa's Grid Engine offering, including options for current Grid Engine users, visit http://www.univa.com/products/grid-engine.php.

About Univa Corporation

Univa, the Data Center Optimization Company, is the leading provider of optimization and management software for traditional, dynamic and cloud data centers. Our award-winning products are used by Global 2500 companies to improve resource sharing, amplify the efficiency of people and processes, and increase application and license utilization. Univa offers the industry's broadest, most innovative and integrated product set for managing shared, high-demand data center resources. From workload management to policy-driven provisioning across physical, virtual and cloud resources, only Univa provides a proven combination of enterprise-class capabilities, industry expertise, and community sponsorship. Univa is headquartered in Lisle, Illinois with offices worldwide.

Intel Cluster Ready is enabling this innovation

The more that things change, the more they stay the same. The fact is that HPC systems will change over time. A newer compute node is likely not a copy-exact replacement of the original failed node, even when acquired from the same manufacturer. Differences can be magnified in larger systems where “MTBF laws” create incidents on a frequent and unpredictable basis (think of Murphy’s law applied to large high-use compute systems).

Often, there are multiple admins and even contractors that provide the care and feeding of the cluster. Many people, many users, many requests and many changes give rise to complexity and can create confusion. Vast amounts of time can be spent figuring out what happened, who did what and when it was done. This reduces efficiency of the system by prolonging downtime of valuable resources, delaying projects and interrupting expensive manpower needed to resolve situations.  Same problem as it ever was, even as individual component technologies improve and become more efficient themselves.

Sometimes you need more…

Package and configuration management unquestionably improve many aspects of maintaining a cluster; however, there continue to be gaps in the operational aspects of sustaining HPTC environments. This is an area of management that Univa has been innovating in partnership with Texas Advanced Computing Center (TACC). With five clusters and nearly 65,000 cores and limited manpower, TACC had to find better ways of managing large-scale clusters. The largest cluster, Ranger, has 3,936 nodes and is managed by both TACC and contract staff. Problem deduction, tracking and resolution have been greatly simplified with a set of operational tools that capture and codify TACC's collective (and impressive) man-years of experience.

Univa's HPC systems management and product development expertise are being leveraged to inventory, generalize and productize these systems. In short order Univa will release substantial improvements to UniCluster's systems management capabilities based on our work with TACC. These innovative systems, best practices and tools will revolutionize HPC systems management and redefine "cluster management."

What is critical to point out is that this innovation and the operational efficiency would not have been possible for a company the size of Univa without a program like Intel Cluster Ready. Since our engineers and product folk are free from solving the same, literally basic problems of provisioning and configuration, we were able to allocate some of our resources to advance the "science" of cluster operations.

Likewise, end-user (like TACC) researchers and admins' time will be freed up to make it possible to accomplish their objectives: that is, to enable science or grand problem solving.

Over the next few posts in my personal blog I will describe these operational tools and systems in more detail. Anyone interested in learning more or previewing the tools may contact Univa at any time. (shameless plug!)

Over the past six years I have discussed management aspects of clusters with a broad cross-section of cluster administrators (large and small), each with their own experiences and views. It was this feedback and perspective that inspired my discussions with Intel and Dell about ISV enablement that ultimately paved the way for Intel Cluster Ready in 2007.

The purpose of the Intel Cluster Ready program was essentially to “level the playing field” for three constituents:

•    ISVs who had to deal with too many incompatible stacks,
•    End-users who faced too many vendor-specific products,
•    The HPC ecosystem that wasted time solving the same problem again and again.

In this post I wanted to highlight what I consider “circular irony”: managing a cluster requires cluster ‘management’. That is, cluster management is more than provisioning, scheduling and packaging some pre-existing tools, and although it seems like middleware vendors don’t seem to get that, there is hope.

It has always struck me as ironic that the various cluster management packages available today rely on a somewhat narrow set of tools that are positioned to comprise all of cluster “management”. Missing, are tools for the operators to actually do something, or affect changes to the cluster and I don’t mean monitoring or staring at consoles. It is, after all, a fact that operating costs typically exceed the capital outlay over the life of the cluster. Sites are generally left to create their own tools or to muddle along forgoing the “network effect” of productized systems software.

Increasing Efficiency of Strategic Assets

High Performance Technical Computing (HPTC) has become linked to an organization’s value chain. Clusters are the dominant architecture in HPTC and these environments have become a strategic element of the product or service, often underpinning an organization’s competitive advantage. For such a strategic asset, one could make a very strong case for the development of a greater amount of efficiency and value!

And Intel Cluster Ready is helping turn that hope into reality.

A key benefit of the Intel Cluster Ready program is that it has allowed Univa engineers to focus on the countless yet-to-be-solved software aspects of the management and operations of a cluster by providing a replicable baseline from which to work.

Cluster users – engineers, scientists, “quants,” analysts and researchers – are interested in the outcome of the science or the software aspects of the cluster. An admin or operator of a cluster is primarily tasked with supporting that: maintaining uptime and handling hardware and software changes as required in support of system use. This involves adding, removing and updating software, installing security patches and replacing failed hardware.

It’s in this stratum that cluster systems software has the greatest room for improvement and thanks to ICR, Univa is able to.

After all, it’s about time.

Lately, I have been paying a lot of attention to the economy watching for signals of any effect on high performance computing spending. The article was based on a report from Gartner in 2008, which in principle flew in the face of the market reality we are seeing, so I pressed on and looked into the story behind the story.

So I pressed on…

“Gartner found spending even in times of economic uncertainty is supported by two factors: businesses are investing in improvements to internal processes aimed at reducing costs, along with their own innovations, and that globalization allows IT services providers to mitigate the risk of weakening demand by operating in more markets.”

And there it was. This reflects exactly what we are hearing from customers – it’s all about “improvements to internal processes aimed at reducing costs.”

To start, let’s frame the circumstance. First and second generation high performance compute infrastructures were built by IT as a service to engineering, research or science. The systems replaced manual processes and were thought of as a means to an end: more computing power. The construct was “throughput”– that is, simply a way to crunch more data in less time. The economic benefit of these generations, usually “get to market faster,” was fairly straightforward.

Over time, improved software modeling developments made possible the ability to simulate more business processes, and that in turn fueled the need for more  memory and compute power. The ability to use COTS components contributed to an explosion in use, size of problems solved and the number of computers in a cluster (also known as a ‘farm’).

After years of unprecedented growth, HPC has evolved from an engineering tool or asset to a far more lofty status including the fastest growing segment of the IT industry in server shipments, roughly one quarter of all CPU shipments; and an invaluable rank in the research, innovation and product development chain of most of the world’s leading companies in virtually every industry.

HPC  underpins product innovation

Having become linked to an organization’s value chain, like the personal computer many years before it, HPC environments are increasingly being viewed differently than in the past. No longer solely the pet projects of visionary CxOs, compute clusters have become a strategic element of the product or service and a competitive advantage. High performance computing has developed from an asset to a value contributor by increasing operating margin of product development through time or efficiency gains and by helping revenue growth through product and service innovation.

That would suggest that the infrastructure has become as visible as other key elements in the value chain, such as manufacturing. That visibility will incur regular lifecycle management inspection as part of business process improvement projects linked to aligning IT spending to corporate goals. Moreover, a greater amount of efficiency and value will be sought from such a strategic ‘asset’. (Similar to how Dell continuously seeks to improve its manufacturing process to improve margins)

Perhaps I should be blunt here. High performance computing is a strategic asset of most (if not all) of the organizations that have deployed it. As such, these organizations have recognized the direct link between investing in the infrastructure and the business rewards (spend more, get more). This is somewhat counter-intuitive to business systems IT where the goal is taking the cost out of the infrastructure.  Thus, processes will need to be developed and improved and a professional and commercial approach will ultimately need to prevail. And it is these developments that will drive requirements back into the ecosystem that seeks to sell solutions to this community.

Seeking Efficiency

So where will this efficiency come from? With the explosion of size and sheer numbers of computers used in a cluster, many organizations have been forced to adapt existing processes or create new processes, including the codifying of workflow, script wrappers and run books. This has only increased the complexity of the environment.  Costs can be driven out of clusters through the development of a sustainable growth model that considers size, complexity, pricing paradigm and inclusiveness. Additional reductions can be realized by employing a systems management software stack that considers the holistic environment and not simply a single aspect of its use. Often, the server:admin ratio can be increased significantly if the maintenance of the system can be offloaded to automation.

Conclusion

Professional management of IT budgets focuses and prioritizes projects on the creation of shareholder value. HPC projects, as we have described, are clearly in this category. However, budget management processes will always seek improvements and efficiencies. IT organizations are reacting and clearly have begun to dictate a new set of requirements to their vendors. The success of the Intel Cluster Ready program is a prime example of this.

I find it amusing if not ironic that organizations have come to expect innovation and more automation from the very software that manages the computing environment that enables their innovation.

An obvious recommendation would be to look for clear signs of innovation from prospective vendors. As vendors implement solutions at different sites their product should reflect the best-of-class ideas of how to manage a cluster. Each site will have very insightful methods or ideas that should be included and made available to a broader set of users. This will allow one to benefit from the ‘network effect’ of improvements in features and functionality across the entire systems management stack.

This ‘network effect’ will drive the most efficiency into the lifecycle management of the cluster. These improvements should allow IT staff to focus on more valuable and strategic projects instead of forcing them to individually learn the same ‘tricks’ and lessons as other organizations every time they push the physical or logical boundary of a system component.

Through this process HPC infrastructures will increase value, become simpler, and IT staff will be able to focus on big picture and interesting business problems. The days of tinkering and DIY may be on the decline.

Who wins? Who loses?

Successful companies face a constant internal challenge. There is a driving need to "Create Value" and its ever-present alter ego "Extract Value." Creating value could include increasing market scope – finding new customers or use cases. However, this is at odds with value extraction, an organization's selfish (understandably so) desire to own and cash in. The challenge organizations face is balancing opening up (give away) and cash-in (take home).

As a principle, then, an "Open Standard" is the goal. I'm not going to define "open standard." Google it and you'll understand why I'm staying away from that. For this purpose I will simply state an open standard is preferable – and it's open IF and only IF it is widely available and adoptable FREE OF RESTRAINT.

Why Open Source?

To be truly open, a specification needs to be free of restraint, meaning free from patent license or copyright use restrictions. Organizations that introduce limitations of either functionality or license unnecessarily hamper innovation and ultimately reduce their own fortunes. But they probably wouldn’t agree with that since their motivation is value extraction and protection (= restriction) of IP is the first step.

Open source affords many of the criteria noted earlier: adoptable, collaborative and generally free of restraint. Community development is also an accelerant. Communities can work at lightning speed and develop quality software. Implementation and incorporation are simplified: engineers are known to be lazy and take the path of least resistance – if something good enough exists, use it….as long as it is "adoptable" and this is where open source shines.

HPC is complex. I haven’t forgot that – but frankly I dismiss that as a crutch. Personally I think there are harder things like learning when to keep your mouth closed in front of your wife so as a to not create more work for yourself (like my stone work, deck and gardening that keeps me busy when the Canadian winter melts into Spring until it returns in the Fall….).

HPC has seen its fair share of openness with many amazing open source projects like Globus, Grid Engine, OFED, Ganglia, Nagios, Lustre to name but only a few. Many of these projects are represented on www.grid.org or included as a component in UniCluster.

HPC has also been the epicenter of commoditization – from engineered performance, proprietary and expensive capability computing machines to off-the-shelf, bundled commodity computing systems available as close to turnkey as possible.

We’ve also had some success, albeit limited, in standardization through the work of many organizations like Open Grid Forum: DRMAA, HPC Basic Profile, BES are some recent specifications.

HPC has taken a BIG step forward through the introduction of Intel Cluster Ready, a specification and program to make it easier to run applications off the shelf. Intel® Cluster Ready is a great example of making complex HPC systems deterministic. In 2005 I set up a meeting with Intel and Dell and we talked about the challenges facing the broad adoption of HPC in savvy and non-savvy commercial organizations. Complexity, engineered performance and commoditization were on a crash course and it was slowing the market, particularly with entry-level customers. We agreed to work together to resolve ISV enablement. The outcome of this ‘French Riviera Summit’ (I was in the South of France during the conference call) was the Intel Cluster Ready program nearly two years later.

A key enabler of this program was open source software. That is, components and reference implementations were readily available because they were open and adoptable. The program would have been hard pressed (and less successful) if elements were closed and unavailable.

Intel Cluster Ready drives out costs.  There are several key factors that lead to this result:

1. Replicability: By creating a programmatic validation process of the systems software components and hardware, there is reduction in time of basic, common repeated steps.

2. Applications working out of the box:  At the end of the day the systems are used to support applications – engineering, research, simulation or modeling. Deploying a system with confidence that the application will work with predictable performance is a significant step forward, especially for the non-savvy and emerging adopters. It also saves time “monkeying around” with the configuration. And time is money.

3. Commoditization is at work:  This occurs when a market has been overshot, suggesting the competitive dimensions are shifting from functionality, performance and reliability to convenience (ease of use), customization (extensions) and price. The fundamental value proposition of Intel Cluster Ready affords more time to software developers like Univa to innovate in functionality that exposes convenience and new untapped capabilities that enable system optimization and lower cost of ownership. By not having to work on the basics we are afforded time to work on these innovations.

As we as an industry progress, increased de-coupling in the components of the solution suggests more interfaces will need to be defined and more convenience brought to the solution to enable new sets of customers to capture the value sooner. Intel Cluster Ready has already done a significant amount of the prescriptive work but there is more work to do. The future for the program is clearly bright.

Most of us credit Ford with the introduction of the assembly line – the “specialization” of the workforce. While true, this is more of an outcome of what Ford did. Ford knew that in order for him to fulfill his vision of a ‘car for everyone’, the way they were made had to change.

At the time every automobile was hand made and practically custom. Many companies were integrated and produced their “own” components. What Ford saw was the opportunity to adopt existing “good-enough” components and incorporate them into his design. Automobiles have a great deal of components: wheels, tires, brakes, engines, starters, seats (to name but a few).

Ford had the benefit of new perspective: he could connect the evolution in the typewriter and clock industry to the manufacture of automobiles. The choices before Ford came down to this: engineer a better automobile using proprietary components or embrace available components and drive towards volume.

The HPC industry, like the clock, typewriter and automobile market before it, is ready for adopting a standardized design and leveraging interchangeable parts. And this is where the inherent value of Intel Cluster Ready comes into play.

And from my point of view – IT’S ABOUT TIME.