Like many of you, I attended SC11 in Seattle this year. I have to say, the world’s largest HPC convention has become downright addictive—especially if you happen to be an ex-gamer. I had to extract myself from the “Gaming through the Decades” exhibition before I could even get started with more serious pursuits. Fortunately my willpower was well rewarded. In case you weren’t there, the focus was on big data and visualization, and the strides being made are truly amazing.

The advances made by the Intel Cluster Ready partner community are also very impressive. We held our third annual Partner Appreciation Award ceremony at SC11, and it was gratifying to see the progress we’ve all made in just the past year. The Intel Cluster Ready architecture is now represented across an enormous range of HPC solutions, from SMB clusters to some of the world’s Top500 supercomputers. Hardware and software support has also grown substantially, as has market penetration. For a look at a few of the most impressive developments, check out the award-winning partners below, and follow the links for more information. I think you’ll be intrigued.

We gave awards in three categories:

• The Pathfinder Award honors the vendor who has done the most to forge new paths into a previously unexplored area.
• Explorer Awards distinguish those who have continued to explore and implement programs that significantly advance the portfolio of solutions based on the Intel Cluster Ready architecture.
• Voyager Awards are for those who have contributed to expanding the global reach and overall market growth of Intel Cluster Ready solutions.

Here’s the list of the 2011 award winners. Enjoy!

Pathfinder Award

• Altair continues to drive innovation by making HPC cluster computing simpler and more productive for its customers. Over the past year, Altair integrated Intel Cluster Checker into PBS Professional, its HPC workload management and job scheduling solution, and registered three key applications for design and analysis: RADIOSS, AcuSolve, and OptiStruct.

Explorer Awards

• science+computing took Intel Cluster Ready into the upper reaches of HPC, installing a 292 TFLOP, 1700 node supercomputer at RWTH Aachen University. It’s the largest Intel Cluster Ready certified system to date, and one of the 30 largest supercomputers in the world. A joint success story will be coming soon to highlight the design win and promote Intel Cluster Ready.
• NEC delivered a huge Intel Cluster Ready win, with a certified 500-node cluster for the University of Erlangen. NEC then helped drive visibility and credibility for Intel Cluster Ready solutions with a success story and Web promotions.
• E4 Computer Engineering launched its E-Cluster 7225 platform, certified Intel Cluster Ready, which scales from four to hundreds of nodes! E4 has also integrated Intel Cluster Ready throughout its marketing communications and updated its website based on each new sales quote.

• Fujitsu has demonstrated a strong commitment to Intel Cluster Ready, integrating support throughout its PRIMERGY server product lines, and have achieved the greatest number of certifications in 2011 for certified Intel Cluster Ready clusters shipped in Europe. Starting at Fujitsu Technology Solutions it became a well-integrated part of the technical production workflow—beginning to end. Intel Cluster Ready is now an integral part of the Fujitsu standard cluster software stack for Intel architecture, and allows them to address a multi-geo market with HPC systems that are 'ready to go'.

• Bright Computing has played an integral role in enabling the Intel Cluster Ready ecosystem by fully incorporating support into its workflows and products, including integration of Intel Cluster Checker into Bright Cluster Manager 5.2. The company was instrumental in helping ensure that the complete HPC solution chain will be ready to support the Intel Xeon Processor E5 family at launch.

• The HPC Advisory Council made broad contributions to Intel Cluster Ready by testing, benchmarking and documenting implementation best practices for open-source applications, co-authoring and publishing a technical brief, and partnering in workshops to extend the awareness and value of Intel Cluster Ready solutions to new audiences.

Voyager Awards

• Flow Science created a cluster solutions guide for Silicon Mechanics and other OEMs to simplify the deployment of Flow-3D, a leading computational fluid dynamics (CFD) solution, on Intel Cluster Ready certified systems. Through this and related efforts, Flow Science is helping to bring HPC—and Intel Cluster Ready—into the digital manufacturing market.
• Silicon Mechanics has integrated Intel Cluster Ready into its technical workflows and sales processes to simplify and accelerate cluster implementation. The company also played a pivotal role in supporting the launch of Sandy Bridge EP and partnered with SIMULIA and Flow Science to deliver optimized, turnkey solutions based on the Intel Cluster Ready architecture.
• SIASA (Soluciones Informaticas Argentinas S.A.) was the first Intel Cluster Ready member in Latin America and has delivered more certified clusters than any other vendor in the region. As one example, SIASA helped the University of Cordoba implement the largest cluster in Argentina.
• Platform Computing is a leading Intel Cluster Ready partner and made a major contribution this year by launching Platform Cluster Manager 3.0, which makes it easier for non-technical customers to build, deploy, and manage Intel Cluster Ready certified systems. The company also provided valuable development support for upcoming Intel server platforms.
• ClusterVision – their commitment to Intel Cluster Ready is so strong that they have edited all their marketing materials and website to promote Intel Cluster Ready. They also implemented Intel Cluster Ready training into their annual sales meeting and launched a sales incentive program. The strategy has delivered new cluster solutions, including a cluster deployment for Ceres Power that was documented in a success story.
• Atipa has shipped eight "Rapid Launch" Linux Clusters in the last two years, all Intel Cluster Ready certified. Atipa has played a valuable role in enabling the HPC ecosystem for the launch of Sandy Bridge EP.
• Eurotech launched its Intel Cluster Ready certified Aurora product line. Then, after collaborating with AuroraScience on a dense, green cluster that’s certified Intel Cluster Ready, Eurotech helped produce a success story about the unique 10 Teraflop Aurora cluster that serves a variety of Italian research organizations and provides a basis for scaling to petaflops of performance.
• Univa has been a long-time supporter of Intel Cluster Ready and recently launched its UniCloud middleware, which makes it easy to deploy Intel Cluster Ready certified HPC clusters in private, hybrid, and public clouds (including RightScale and Amazon EC2).

That’s it for this year’s winners. I’d like to add my personal thanks to all of them.

Stay tuned for more information. I’ll be focusing in on some of these technologies and solutions in more detail in future blogs.

Intel Cluster Checker can run a variety of tests depending on what the user is trying to accomplish. For general wellness of a cluster, Intel Cluster Checker offers five levels of thoroughness. For automation, we'll focus on levels one, three, and five.

Wellness Level One is a very short run of tests that check basic connectivity throughout the cluster as well as basic uniformity checks amongst the nodes.  This test focuses on BIOS settings, and processor, memory and system configurations. The level one tests are quick and show that the cluster is online and ready for use.

Wellness Level Three is the default run level for Intel Cluster Checker. It builds on level one and includes more rigorous modules that test parameters like disk and memory bandwidth, MFLOPS, and network performance. It also performs an in-depth hardware uniformity test along with an Intel MPI Collectives and message Integrity test. It takes a bit longer to run this check that assures the performance of the hardware is up to par.

Wellness Level Five adds a packages test, comparing currently installed packages with a generated list of expected packages at a given time. It also runs the HPCC module (a performance benchmark test). The level five tests can take a bit longer but assures the users that all the pieces of the system are working in harmony. This test is useful for the admin to make sure a user hasn’t installed or uninstalled anything that may affect the cluster.

For my clusters, I have set up the following schedule that runs these wellness levels at various times.

• Level One - set to run each weekday
• Level Three - set to run once a week, on Saturday
• Level Five - set to run monthly, on the first Sunday

For our lab with multiple smaller clusters this automation schedule is perfect, allowing users to login to one of the clusters, see when each of the wellness levels passed and know that their jobs will run as expected.

Every cluster has a different purpose and a different load. Running tests at the intervals or frequency I have set may not be feasible or needed for every cluster. However, automating the test runs on a schedule that meets your needs will help assure that your cluster performance remains optimal, and frees up the cluster(s) for jobs during normal business hours.

References:

>> Review Part 1

>> About Intel Cluster Checker

>> Intel Cluster Checker Knowledge Base

Whether you're a cluster user that expects optimal performance and functionality every time you run a job, or a system administrator that needs to keep the cluster in perfect working order for your users, running checks on a regular basis is important!

A great way to ensure that your HPC system, certified Intel Cluster Ready, remains in the same great shape as when it was first built, is to run the Intel® Cluster Checker tool regularly.

While it’s easy enough to run Intel Cluster Checker on your cluster(s) once a week, I've found there are times when I was too busy, or simply forgot to run a check. For this reason, and because I have multiple clusters to check, I developed a method to automate Intel Cluster Checker runs. This automated solution runs Intel Cluster Checker and reports any errors directly to me and/or the system administrator. Not only does automating this process save the administrator a substantial amount of time each week, it will eliminate the chance of missing a run, and ensures that the cluster remains in optimal health.

A Cluster That Checks Itself

In order to automate the process, it is important that your Intel Cluster Checker passes a manual run first. To do this, make sure the configuration file is optimized to ensure the cluster is operating at its best. With the perfect configuration on hand, a script is then needed to setup the system, run Intel Cluster Checker, and report the results.

For the first phase of the process, I wrote a script that sets up the environment and runs the most in-depth wellness check in Intel Cluster Checker. I used cron to schedule my script to execute once a week. After the results are complete, the script updates the message of the day to show when the last check ran and the results. If the check fails, the log file created by Intel Cluster Checker will be copied to the specified directory where it can be accessed and analyzed.

The cluster is now setup to check itself, in the middle of the night, and report its status upon login!

Read Part 2 for more about scheduling test runs, and the different levels of wellness tests in Intel Cluster Checker.

References:

>> About Intel Cluster Checker

>> Intel Cluster Checker Knowledge Base

Seamless Transitions...

I'd add that the Intel Cluster Ready architecture applies in the cloud as well.  In fact, using cloud images based on Intel Cluster Ready architecture allows more seamless transitions between clusters running in the cloud to physical clusters running locally.  As the video points out, cloud provides an entry point into using HPC or even a try-before-buy approach to HPC clusters.  Cloud bursting benefits as well, since the application that runs on the architecture locally would see the same architecture in the cloud.

The common architecture enables the applications to better understand and execute on top of the computing environment without being tied to the exact details of the solution.  In other words, you can get better mileage with a common architecture in both the cloud and local clusters.

While manning the Intel Cluster Ready demonstration pod, I was able to listen in on Drew and Kirk talk about how real-time simulations are helping protect football players from serious injury.  They showed how helmets outfitted with sensors are being used to study and relay data about individual impacts to the head of players.  Simulations for those impacts can predict when dangerous levels of force have occurred and players are likely to have suffered a concussion.

This data can be used to help design better equipment, but more importantly may help prevent serious, long-term injury.  Players who have experienced a concussion may not feel the immediate effects or may not want to leave the game.  Additional impacts, however, can cause much more severe injury.  The simulation data can help identify when players need to be pulled out of a game for medical evaluation and/or to avoid serious injury.

It was a really cool demonstration to see - they even cracked two helmets together to show a live simulation of impact. One of many neat examples of high performance computing in action that I got to see at the show.

>> See related stories

Desktop Engineering: Intel to Safeguard Brains

Intel Newsroom: Intel Teams Up with Leading Researchers to Make Football Helmets Safer

Were you there? Please leave your comments about SC10.

Check back for more news and a follow-up from SC10.

Let Intel Cluster Ready be your guide…

Take the Intel Innovation Tour, learn about some of the Intel Cluster Ready partners, and enter to win a HP Pavilion Mini Netbook. Several tours each day, and a daily drawing at 5:00pm Wed. Check at Intel booth #2813 for times and details.

In-Booth Presentations & Demos at Intel Booth #2813
Come by to see the demos, hear about the latest Intel technologies, and catch up with the Intel Cluster Ready team and partners.

Presentation:

• Wednesday, Nov. 17, 10:00 am – Intel's Bill Magro will present Everything Works! End User Experience with Intel Cluster Ready. Come hear about end users like Simpson Strong-Tie and others who are reaping the value of an Intel Cluster Ready cluster.

Demos:

• 7 Reasons Why You Need Intel Cluster Ready, by Aspen Systems, Inc., Professional HPC Clusters and Services
• Appro Ready-To-Go Cluster Demo: Ready-To-Go Clusters for Seismic Coverage Validation Engine, part of Halliburton, Landmark ProMAX® software suite - Maximize geophysical investments with Appro Ready-To-Go Cluster, a pre-configured Intel® Cluster Ready certified supercomputing solution. Pre-tested to run Landmark's new Seismic Coverage Validation software tools, this solution makes the process of choosing the right system easy.

More Fun Events to Come at SC10!

HPC Idol 2010 – Giving up those HPC blues. Have time for a party on Wednesday evening? Don't miss the Platform Computing party at the House of Blues, from 6:30 – 10:00pm, Nov 17th.

Possibly, one of the highlights of this event will be an original, never before and likely never again, song from Intel Cluster Ready.... "Get Ready" with back-up vocals by the "Clusterettes."

How do you get your invitation to attend? Just stop by the Platform Computing booth #2739, Intel booth #2813, or QLogic booth #3649 to ask for an invitation to attend!

Partner Presentations

In the Appro Booth #1939:
Wednesday Nov 17th -

• 11:15am – Intel Cluster Ready and Appro Ready-To-Go Clusters, Rick Layden, Intel

• 11:45pm – Ready-To-Go Clusters with ANSYS, Barbara Hutchings, ANSYS

In the QLogic Booth #3649:
Wednesday Nov 17th -

• 1:30 pm – It's all about InfiniBand Clusters, presented by Jeremy Siadal of Intel.

Don't forget to look for us, your Intel Cluster Ready team, in booths 2813 and 4517.

Hope to see you here in New Orleans!

View webinar » Boost ANSYS Performance with HPC Clusters

On October 28th, ANSYS will host a joint webinar with Appro and Intel that highlights using ANSYS Mechanical HPC on clusters and dives into details of how the three companies are working to address issues to make clustering simpler.

I'll be participating in the talk and providing examples of what Intel Cluster Ready architecture means to the ANSYS application and the Appro solution.  Together, we'll focus on points that relate directly to our efforts to simplify purchasing, deployment, management, and use of cluster solutions.

This Virtual Seminar will be interactive.  We want to hear what your pain points are related to clusters, too! 

Please join us and take part in the discussion. Choose the AM or PM session that works best for you. Click on the headline below to access the registration links.

Webinar: Boost ANSYS Performance with HPC Clusters

Thursday, October 28, 2010
9:00 A.M. EDT, 1:00 P.M. GMT

Thursday, October 28, 2010
1:00 P.M. EDT, 5:00 P.M. GMT

Duration: 90 minutes

Join us for a Virtual Seminar to learn:

• How to simulate, analyze, and visualize models faster by deploying ANSYS HPC
• How the ANSYS and Intel Cluster Ready partnership simplify purchasing and speed deployment, helping to make cluster management affordable and easy
• How Intel Cluster Ready can help you to deploy, troubleshoot and optimize an ANSYS HPC cluster

All so you can get your job done faster.

Register now!

WHO: Halliburton's (NYSE: HAL) Landmark and Software Services business line, premier provider of software and technology for the upstream oil and gas industry, and Appro, a leading provider of supercomputing solutions, and Intel, a world leader in computing innovation

WHAT: Announce the launch of an Appro pre-configured Intel® Cluster Ready certified supercomputing solution that is pre-tested to run Landmark's new Seismic Coverage Validation Tools.  The wave equation-based Illumination and Visibility software helps geoscientists design 3D seismic surveys and select processing parameters optimized for geologic targets in traditionally poorly imaged areas. The software runs on a pre-configured Appro GreenBlade™ System based on Intel® Xeon® processors featuring dual socket, 32 nodes with up to 12 CPU cores per node and 8GB of memory. The result provides geoscientists with an Intel® Cluster Ready solution that is easy to deploy and maintain "out of the box."

WHEN: Demonstrations will occur at the SEG International Exposition October 17 – 20 in Denver, CO.

WHERE: Appro booth #1116, Halliburton booth #1611, and Intel booth #2562.

WHY: Complex subsurface environments, like deepwater and subsalt, present many challenges to seismic acquisition design and imaging, including azimuthal orientation, image shadows and amplitude variations in the data which can lead to incorrect interpretations and reservoir models. Landmark's Illumination and Visibility technology analyze and model the seismic signal from source, to subsurface target, to receiver so that the geophysicist can optimize imaging parameters to a specific subsurface target or make the required design changes to future seismic acquisition surveys. These accurate wave equation-based techniques are compute intensive, requiring the high performance computing from Appro's cluster deployment. This architecture, optimized to run Landmark's Seismic Coverage Validation tools, will provide faster results so geoscientists can make improved and more timely exploration decisions while lowering overall exploration costs.

For more information please visit www.halliburton.com.

About Appro
Appro is a leading developer of supercomputing solutions. Appro is uniquely positioned to support High Performance Computing markets focusing on small, medium to large-scale deployments where lower total cost of ownership is essential. Appro accelerates technical data-intensive applications for faster business results through outstanding price/performance, balanced architecture coupled with latest technologies, open standards and engineering expertise. Appro headquarters is in Milpitas, CA with offices in Korea and Houston, TX.

To learn more go to www.appro.com.

About Landmark
Landmark is the premier provider of software and technology services for the upstream oil and gas industry. Our software solutions, built for the DecisionSpace® environment, help improve insight from data in ways never possible before, across the entire exploration and production life cycle. Landmark's technology deployment and hosting services, petrotechnical computing portfolio, software training, and certified customer support are globally available to help our customers realize the maximum return on their technology investments.

Visit www.halliburton.com/landmark/ for more information.

About Halliburton
Founded in 1919, Halliburton is one of the world's largest providers of products and services to the energy industry. With more than 55,000 employees in approximately 70 countries, the company serves the upstream oil and gas industry throughout the life cycle of the reservoir - from locating hydrocarbons and managing geological data, to drilling and formation evaluation, well construction and completion, and optimizing production through the life of the field.

Visit the company's Web site at www.halliburton.com

Ultimately, it's about solving some problem. The size of the problem can dictate needed features or characteristics in a cluster.  It's a logical tendency, then, to take an application and build a cluster from scratch that meets the needs for that specific application. The problem is that this implies customized clusters every time.

A common cluster platform architecture allows solution designs that are flexible to support a wide spectrum of applications, while still providing the appropriate hooks to adapt to the specific needs of the application workloads.

What does that mean for the broader span of would-be cluster purchasers? That decisions of what to buy converge more towards the vendor's quality of solution, plus the configuration needs of the workloads. Then the purchaser doesn't have to understand as much about how the system works under the hood.

The relationship is very important between the cluster solutions vendor, the cluster purchaser, and the application vendor. Knowing how faster processors, huge amounts of memory, InfiniBand interconnect, or twice the number of compute nodes will reap benefits is a non-trivial question. Expecting that cluster purchasers will know the answers themselves, especially those just coming into using HPC, is not feasible. The applications vendors need to help provide the guidance here, allowing cluster solutions vendors to provide the tunable configurations for cluster purchasers to select what fits their computing needs, and budget.

To ease the purchase process, many solutions vendors and applications vendors are using Intel Cluster Ready architecture to form partnerships.  Configuration guides from the applications vendor can help point purchasers to suggested solutions based on the size of workload. Then the solutions vendor offers systems that meets the need of the workload size. Together they provide fully specified solutions for the purchaser to select.

Applications built from source are a bit tougher because of the difficulty in forming a relationship to solutions vendors, but it's still possible to achieve the same goal. The application source code can provide a build environment for the common architecture. The source code maintainers can provide guidance on what software tools are needed or desired and how solution components affect performance of workloads. This in turn allows solutions vendors to match offerings to the needs of the application.

How fast the use of HPC expands will partially depend on how easy it becomes for new users to match solutions and applications. Building from scratch every time will dampen that expansion. New users will need to have guidance from both the cluster solutions vendor and the applications vendor or source code maintainer to make it easier and more attractive to jump into HPC.

What do you think?

>> See more Related Stories, and check out the discussion at:

Cluster buyer vs. seller: do-it-yourself builds

Volume HPC cluster solutions need to move away from cafeteria selection of components to pre-defined, flexible designs that meet a broad spectrum of user needs. This will take a lot of complexity out of purchasing HPC clusters and remove the need for the customer to be a cluster systems expert. I'm not suggesting that cluster purchasers have zero understanding of the system, but it needs to be simplified to understanding the application workload needs that will run on the system.

Purchase time decisions for volume cluster solutions need to center on aspects that affect execution in terms of performance but do not risk introduction of component integration issues.  That means that components like operating system, cluster provisioning, interconnect hardware, and resource management software need to be set and unchangeable at purchase time.

Allowing total piecemeal selection of components at purchase time requires the buyer to know how everything fits together, have the tools and knowledge to build the system, and/or pay for the services to put it together.  This approach works for veteran buyers who easily justify the return on investment but is much harder for the (new) volume cluster buyer.

Volume cluster solutions are more like partnerships of many components with the solutions provider as the focal point.  Solidify the solution component partnership, and the solutions themselves become easier to produce and ultimately become more stable product. That helps lower barriers to using HPC clusters.

There's a key set of partners I haven't covered yet, though, the application vendors themselves, but that's another topic all together.

>> See Related Stories for more about how to simplify selecting, buying, and managing HPC clusters... and come back for more from this weekly blog.