Boverhof’s App Earns Honorable Mention in Amazon’s Web Services Competition

The Turbine Gateway, an application developed by the Computational Research Division’s Joshua Boverhof for DOE’s Carbon Capture Simulation Initiative, recently earned honorable mention in a competition sponsored by Amazon Web Services (AWS). Amazon officially announced the winners of its EC2 Spotathon on Monday, Dec. 10.

“We received many impressive applications and there was fierce competition at the top; even though we ultimately didn’t select your entry for the Grand Prize or Runner-Up, given the caliber of your Spot application, we will be recognizing the Turbine Science Gateway as one of the honorable mentions on our web page and in an upcoming blog post,” Amazon Web Services Senior Product Manager Stephen A. Elliott wrote to Boverhof. “Congratulations on the honorable mention and, more importantly, for developing such a sophisticated Spot application!”

Boverhof learned of his award at an AWS Reinvent developers conference a few weeks ago and really didn’t expect to win any recognition, but was quite happy with the result.  He showed that a single experiment that ran for 11 hours saved more than $300 by bidding on unused compute resources on the spot market versus paying for on-demand instances.

Amazon set up the Spotathon “to publicly recognize some of the most exciting applications of EC2 (Elastic Compute Cloud) Spot Instances, assessing each submission based on the cost savings, performance benefits, and compute scale achieved on Spot, as well as the overall elegance of the solution.” EC2 provides scalable, pay-as-you-go compute capacity in the cloud. 

Spot Instances are excess EC2 instances whose price is based on real-time supply and demand. When users request an EC2 Instance via a Spot request, they specify a bid price; as long as the bid exceeds the Spot price, the user is provisioned a Spot Instance. When the Spot price exceeds the bid, the Spot Instance is interrupted.

In its announcement, Amazon stated “The Turbine Science Gateway (TSG), developed by Joshua Boverhof, supports the Department of Energy’s Carbon Capture Simulation Initiative (CCSI) by providing a web application and execution environment for running and managing scientific applications (like AspenTech’s AspenPlus) and storing and archiving results. Utilizing TSG, simulation runs that would take months on a single machine can be done overnight on EC2, running tens of thousands of simulations on hundreds of Spot instances and saving over 70 percent on EC2 compute costs.”

This past June AWS announced the integration of Spot instances with their auto scaling capabilities. Boverhof integrated that into TSG so the application could automatically take advantage of additional resources as they became available. By the time he submitted his entry, Boverhof and the CCSI team had run more than half a million simulations, giving him plenty of data on which to build his case.

When submitting his entry, Boverhof described a recent experiment of a hybrid circulating fluidized bed process model, in which the CCSI project’s Uncertainty Quantification (UQ)  team of scientists and engineers at Lawrence Livermore National Lab ran 50,000 simulations utilizing 240 spot instances. The experiment ran overnight, and took a total of 11 hours to run.  Boverhof states in his submission, “While the experiment was running with 240 spot instances, we were paying between $7.84 and $12.78/hour for EC2 instances. Running the experiment on-demand would have cost $41.94/hour for EC2 instances. Total saving on just this experiment was over $300.”

The TSG is a web application and execution environment for running and managing scientific applications and storing and archiving results. It’s part of an integration framework, developed to support application scientists in the Carbon Capture Simulation Initiative (CCSI), a partnership among DOE laboratories, industry and academic institutions. The project aims to provide the utilities industry with advanced tools to further leverage computational modeling and simulation tools to accelerate the use of carbon capture technologies from discovery to development, and ultimately widespread deployment to hundreds of power plants.