Texas Unleashes Stampede for Science

By Aaron Dubrow, Texas Advanced Computing Center
Published: March 27, 2013

Texas Advanced Computing Center’s latest supercomputer powers transformative discoveries across science and engineering

You hear it before you see it — a roar like a factory in full production. But instead of cars or washing machines, this factory produces scientific knowledge.

Stampede, the newest supercomputer at the Texas Advanced Computing Center (TACC) and one of the most advanced scientific research instruments in the world, fills aisle after aisle of a new 11,000-square-foot data center on the J.J. Pickle Research Campus. Through the glass machine room doors, you can see 182 racks holding more than 500,000 interconnected computer processors. Inside, wind whips from in-row coolers, wires snake over the racks and chilled water courses below the floor as Stampede performs calculations on behalf of scientists and engineers nationwide.

Stampede Research Bits

Check out examples of research already underway using Stampede’s supercomputing power.

( Click on images )

 

 

 

  Getting Beneath the Tip of the Iceberg
Omar Ghattas’ team at The University of Texas at Austin is using Stampede to better understand and represent the flow of ice from Antarctica into the sea using detailed numerical models.

Predicting the Big One
Thomas Jordan, of the Southern California Earthquake Center, is using Stampede to forecast the frequency of damaging earthquakes in California.

Analyzing Audio
Researchers in the social sciences, digital humanities and arts are using Stampede to enable new discoveries. Stampede provides high-performance computing and large-scale visualization to sound archivists to help them search for patterns and gain insights into spoken language and music.

Over the past year TACC staff designed, built and deployed Stampede, working closely with Dell and Intel engineers and university researchers. TACC and The University of Texas at Austin competed against the top supercomputing centers and universities to claim one of the most advanced systems in the world — and won. The award was funded by the National Science Foundation (NSF) with an estimated investment of more than $50 million over a four-year period. The project may be renewed in 2017, which would enable four more years of open science research.

According to the November 2012 Top 500 list of supercomputers, Stampede is the seventh-most powerful advanced computing system on the planet and the most powerful in the U.S. dedicated to academic research, capable of outperforming 100,000 home computers.

On Wednesday, March 27, leaders from government, academia and industry, including The University of Texas at Austin’s President Bill Powers, Jay Boisseau of TACC, Marius Haas of Dell, Diane Bryant of Intel, Farnam Jahanian of the NSF and U.S. Congressman Lamar Smith, will dedicate Stampede and kick off a new era of advanced computing at the university and nationally.

“This is a proud moment for The University of Texas,” President Powers says. “Stampede will serve UT and the nation as it enables scientific exploration that would not otherwise be possible, and it continues TACC’s tradition of providing powerful, comprehensive and leading-edge advanced computing technologies to the open science community.”

As its name suggests, Stampede harnesses the power of a half-million computer processors and combines them to tackle ever larger and more challenging computational problems. Sixteen times more powerful than the recently decommissioned Ranger system (which was most recently ranked as the 50th-fastest supercomputer in the world), Stampede will enable scientists to address new classes of problems they’ve never been able to approach before.

“How often does a scientist get an instrument that’s an order of magnitude more powerful than the one it replaces?” says Omar Ghattas, the John A. and Katherine G. Jackson Chair in Computational Geosciences. “It’s a massive step.”

In recent years, supercomputers have become critical general-purpose instruments for conducting scientific research. Known as the “third pillar” of science, computer simulations and models complement theory and experimentation and allow researchers to explore phenomena that cannot be captured via observation or laboratory experiments.

Supercomputers also allow scholars to mine massive databases of information for digital needles-in-haystacks that otherwise would go unnoticed — such as subtle changes in DNA or the signs of a newly discovered galaxy in a far corner of the universe.

Closer to home, the weather report you checked before going outside, the car you drove to work, the flu shot that protected you from illness — all of these were, at least in part, designed, improved or predicted by a supercomputer.

The reason supercomputers are so important is simple: The universe is governed by mathematical equations, and computers can solve these equations far faster than humans. Enormous supercomputers, like Stampede, enable researchers to solve scientific problems that humans along would find impossible — the kind that help predict where a hurricane will make landfall, how a new drug will interact with its target, or what mutations in our genetic code make us prone to developing certain diseases.

Stampede acts as a “computational microscope” that allows scientists to explore the inner dynamics of the cell better than with the best imaging devices; helps astronomers peer deeper into the universe’s past than is possible with the most powerful telescopes; enables researchers to develop new materials to remove CO2 from the atmosphere; identifies brain tumors more accurately; and discovers new medicines faster and less expensively than in a laboratory. Stampede will even help researchers in the digital arts and humanities study literature and music in ways they never imagined before.

In the past, supercomputers were often used for a small subset of science and engineering problems. But systems like Stampede, with its comprehensive capabilities, allow many more users to simulate, visualize, analyze, store and share their knowledge with others around the world.

In the first three months of operations, approximately 600 projects and more than 1,200 scientists have used Stampede, which came online in January 2013. These include top researchers in every field of inquiry from mechanical engineering to linguistics to neuroscience. In its lifetime, Stampede is expected to deliver the equivalent of more than 400,000 years of computing to tens of thousands of scientists. Imagine the results Stampede will enable across all fields of knowledge.

Read more about research already underway using Stampede:

Improving Brain Tumor Imaging

The Chemistry of Water

A New Era in Computational Biology

Carbon Dioxide Capture and Conversion

_____________________________________________________________

_____________________________________________________________

Aaron Dubrow (2013).
Texas Unleashes Stampede for Science
UT – Texas Advanced Computing Center

_____________________________________________________________

View the original article here

A great role model for collaborative science: meet the OpenWorm

A great role model for collaborative science: meet the OpenWorm
A couple of weeks ago I had the pleasure to meet the team of OpenWorm in two live meetings that the they held in the UK.

– from their website:

OpenWorm aims to build the first comprehensive computational model of the Caenorhabditis elegans (C. elegans) [..] With only a thousand cells, it solves basic problems such as feeding, mate-finding and predator avoidance. [..] 

We are using a bottom-up approach, aimed at observing the worm behaviour emerge from a simulation of data derived from scientific experiments carried out over the past decade. [..] We are engineering Geppetto, an open-source simulation platform, to be able to run these different models together. We are also forging new collaborations with universities and research institutes to collect data that fill in the gaps.

Rigorous predictive models are the cornerstone of science and engineering. Predictive models took us to the moon. Unfortunately, today, there are no comprehensive predictive models of living cells and tissues. Consequently, the entire field of biology and medicine is in a kind of “pre-mathematical” era. A revolution in the biosciences driven by simulation-based research, using predictive models running on high performance computing architectures with flexible user interfaces, is now possible. Simulating a living organism will have impacts on understanding mechanisms of disease, drug discovery and development, synthetic biology, bioengineering, and neuroscience. This is so important to us. 

+Matteo Cantarelli - member of the OpenWorm team - says: "It was 2007 when +Giovanni Idili and I started - naively - talking about simulating the worm. We were approaching the problem after having hit common limits of artificial intelligence. We never got to write any code for the worm at that time, we just had lengthy conversations and paper reading sessions together."

At that time +Stephen Larson was involved in an open-source, 3-D virtual environment project developed by a team of researchers from UC San Diego: the Whole Brain Catalog  About two years after that attempt to simulate the worm with his friend Matteo, Giovanni Idili tweeted on the Whole Brain Catalog's account, these words: "new year's resolution: simulate the whole C. elegans brain (302 neurons)!" - Stephen got the message and for a while he thoroughly pondered this idea.
In fact, Stephen gave a talk @IgniteSanDiego in 2010 where he officially launched the idea. He stated that if we want to get a really high-level A.I. (like Star Trek's Lieutenant Data), we should probably start with a simpler organism.  Even though it is “simpler”, reproducing the functions of C. elegans in a simulation reveals its complexity. The hermaphrodite form has just 302 neurons and the connectome has been already mapped. After this talk, a lot of people were excited about this idea, so Stephen proposed a survey to reach people around the world who were already interested in this subject. "The key was to find people that were already working on worms and convince them to work together. There are plenty of people around that are excited to do what you have in mind, you just have to find them and show them that working together is better than working separately.." Matteo and his friend Giovanni enthusiastically replied that they wanted to "make the worm". At that point, a series of Skype calls began where the initial team was figuring out the best way to reach this goal. Stephen says: "A good resource that I found useful was  Karl Fogel's Producing Open Source Software free book, which, beyond coding, shows you how to deal with getting started, permission statements, transparency, and so on.. I've learned a lot from this book, and the choice of the name - OpenWorm - was meant to be simple and understandable as they suggest.. "

"There's plenty of science that still happens in non-english language, so that there's still a lot of talent that isn't talking to the rest of the world."

Another serendipitous event was the finding of a YouTube video of a team in Siberia - Andrey Palyanov and Sergey Khayrulin - who had taken the neurons of C. elegans and created a virtual physical body. The team contacted them through the YouTube messaging system because at that time there were no references on the video. At the end they also joined the team. Stephen: "I realized that there's a lot of scientific communities within Russia and outside Russia that is still rather disconnected. There's plenty of science that still happens in non-english language, so that there's still a lot of talent that isn't talking to the rest of the world. So for us, that was a really great find.."

Then Google +Hangout came in, so they had a broader audience and it was easier to handle conference calls with more people.
They currently meet every two weeks on Google hangout. A great practice they have is to discuss high profile papers related to the simulation of biology with the authors, and stream the talks on YouTube. You can take a look on this Journal Club on their channel. Stephen, Giovanni, Matteo and Mike @+OpenWorm Live in Cambridge, UKMatteo: "At the beginning it was Stephen, Giovanni, Tim and I, if I remember correctly. First, I started building an ontology with Stephen to try identify the high-level requirements for simulating the worm. What was needed, what physical aspects we had to simulate and to gain a better understanding of the big picture. After that I worked to transform the requirements we gathered into technology by designing the architecture of the simulation engine, what we recently named Geppetto, and took a first stab at implementing the different modules. During this time Giovanni was writing the first module for the simulation engine with a proof of concept of a simple Hodgkin-Huxley neuronal simulator that leveraged GPU".Stephen: "Other people eventually joined the project. Christian Grove from Caltech who contributed with the 3-D rendering of the worm. Then, a group from INCF,  from which we decide to use NeuroML, a standard language for the model, so then we get connected to the NeuroML community where more people came to the project." He also stressed the concept to have an organizational structure, for instance OpenWorm is structured like the Bazaar theorized by Eric S. Raymond, where there's a central coordinating platform but it’s actually run by a series of sub-project that has a common direction.

“Chance handed me a perfect way to test my theory, in the form of an open-source project that I could consciously try to run in the bazaar style. So I did—and it was a significant success.” -- from The Cathedral and the Bazaar by Eric S. Raymond

"...you start to write down on a piece of paper all the things that comes into your mind, and you try to connect them, and all the aspects between them. When you do such a brainstorming you might realize that some stuff are related, and that this and that could be grouped together. So, if you're creative enough, you start to find some larger theme that cover all this stuff, and that gives you the ability to work on all the things you're actually interested in at same time..."

Stephen gives other suggestions to people that wants to commit themselves in a collaborative project. I think that his suggestions can be true for - pretty much - any young scientist who wants to approach this kind of career without being disappointed at the end of the day. In my opinion this could really work in general for any scientist.  "I think you got to find something that you're truly excited about. [..]  I think that the challenge early on is that is really easy to be excited in something for about two months and then you give up. It means that probably you're not really interested in it, or you can try to discard this idea for a while and return back on it later. Sometimes stuff works better later."
So, after you have done your homework, you start to write down on a piece of paper all the things that comes into your mind.  When you do this brainstorming you might realize that some things are related. [..] and you may start to find some larger theme that covers all this stuff.  That gives you the ability to work on all the things you're actually interested in at same time. For me OpenWorm turned into that. I was interested data visualization, artificial life simulation, neuroinformatics, something that is biologically relevant... but there was a lot of things in my previous projects that didn't actually allowed me to stretch into all these topics. OpenWorm was for me the key project to focus on all the things I was truly excited about at the same time."

OpenWorm is made of many other people, most of which aren't mentioned here. But all of them are incredibly committed to what they're doing. I think that this is one of the best projects to join if you'd like to work with skilled people with a genuine passion for Artificial Intelligence. OpenWorm is definitely something that is truly visionary. Part of the OpenWorm team and fans @+OpenWorm Live in London, UK

"All the code we produce in the OpenWorm project is Open Source and available on GitHub. This makes it easy for everyone to look at and play with.Detailed documentation for each sub-project is available from the Projects page."

Varshney LR, Chen BL, Paniagua E, Hall DH, & Chklovskii DB (2011). Structural properties of the Caenorhabditis elegans neuronal network. PLoS computational biology, 7 (2) PMID: 21304930

Palyanov A, Khayrulin S, Larson SD, & Dibert A (2011). Towards a virtual C. elegans: a framework for simulation and visualization of the neuromuscular system in a 3D physical environment. In silico biology, 11 (3-4), 137-47 PMID: 22935967

View the original article here