Who’s the most influential in a social graph? New software recognizes key influencers faster than ever

ScienceDaily (Sep. 7, 2012) — At an airport, many people are essential for planes to take off. Gate staffs, refueling crews, flight attendants and pilots are in constant communication with each other as they perform required tasks. But it's the air traffic controller who talks with every plane, coordinating departures and runways. Communication must run through her in order for an airport to run smoothly and safely.

In computational terms, the air traffic controller is the "betweenness centrality," the most connected person in the system. In this example, finding the key influencer is easy because each departure process is nearly the same.

Determining the most influential person on a social media network (or, in computer terms, a graph) is more complex. Thousands of users are interacting about a single subject at the same time. New people (known computationally as edges) are constantly joining the streaming conversation.

Georgia Tech has developed a new algorithm that quickly determines betweenness centrality for streaming graphs. The algorithm can identify influencers as information changes within a network. The first-of-its-kind streaming tool was presented this week by Computational Science and Engineering Ph.D. candidate Oded Green at the Social Computing Conference in Amsterdam.

"Unlike existing algorithms, our system doesn't restart the computational process from scratch each time a new edge is inserted into a graph," said College of Computing Professor David Bader, the project's leader. "Rather than starting over, our algorithm stores the graph's prior centrality data and only does the bare minimal computations affected by the inserted edges."

In some cases, betweenness centrality can be computed more than 100 times faster using the Georgia Tech software. The open source software will soon be available to businesses.

Bader, the Institute's executive director for high performance computing, says the technology has wide-ranging applications. For instance, advertisers could use the software to identify which celebrities are most influential on Twitter or Facebook, or both, during product launches.

"Despite a fragmented social media landscape, data analysts would be able to use the algorithm to look at each social media network and mark inferences about a single influencer across these different platforms," said Bader.

As another example, the algorithm could be used for traffic patterns during a wreck or traffic jam. Transportation officials could quickly determine the best new routes based on gradual side-street congestion.

The accepted paper was co-authored by Electrical and Computer Engineering Ph.D. candidate Rob McColl.

Share this story on Facebook, Twitter, and Google:

Other social bookmarking and sharing tools:

Story Source:

The above story is reprinted from materials provided by Georgia Institute of Technology.

Note: Materials may be edited for content and length. For further information, please contact the source cited above.

Note: If no author is given, the source is cited instead.

Disclaimer: Views expressed in this article do not necessarily reflect those of ScienceDaily or its staff.

View the original article here

Writing graphics software gets much easier: New programming language yields code that’s much shorter and clearer -- but also faster

ScienceDaily (Aug. 1, 2012) — Image-processing software is a hot commodity: Just look at Instagram, a company built around image processing that Facebook is trying to buy for a billion dollars. Image processing is also going mobile, as more and more people are sending cellphone photos directly to the Web, without transferring them to a computer first.

At the same time, digital-photo files are getting so big that, without a lot of clever software engineering, processing them would take a painfully long time on a desktop computer, let alone a cellphone. Unfortunately, the tricks that engineers use to speed up their image-processing algorithms make their code almost unreadable, and rarely reusable. Adding a new function to an image-processing program, or modifying it to run on a different device, often requires rethinking and revising it from top to bottom.

Researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) aim to change that, with a new programming language called Halide. Not only are Halide programs easier to read, write and revise than image-processing programs written in a conventional language, but because Halide automates code-optimization procedures that would ordinarily take hours to perform by hand, they're also significantly faster.

In tests, the MIT researchers used Halide to rewrite several common image-processing algorithms whose performance had already been optimized by seasoned programmers. The Halide versions were typically about one-third as long but offered significant performance gains -- two-, three-, or even six-fold speedups. In one instance, the Halide program was actually longer than the original -- but the speedup was 70-fold.

Jonathan Ragan-Kelley, a graduate student in the Department of Electrical Engineering and Computer Science (EECS), and Andrew Adams, a CSAIL postdoc, led the development of Halide, and they've released the code online. At this month's Siggraph, the premier graphics conference, they'll present a paper on Halide, which they co-wrote with MIT computer science professors Saman Amarasinghe and Fredo Durand and with colleagues at Adobe and Stanford University.

Parallel pipelines

One reason that image processing is so computationally intensive is that it generally requires a succession of discrete operations. After light strikes the sensor in a cellphone camera, the phone combs through the image data for values that indicate malfunctioning sensor pixels and corrects them. Then it correlates the readings from pixels sensitive to different colors to deduce the actual colors of image regions. Then it does some color correction, and then some contrast adjustment, to make the image colors better correspond to what the human eye sees. At this point, the phone has done so much processing that it takes another pass through the data to clean it up.

And that's just to display the image on the phone screen. Software that does anything more complicated, like removing red eye, or softening shadows, or boosting color saturation -- or making the image look like an old Polaroid photo -- introduces still more layers of processing. Moreover, high-level modifications often require the software to go back and recompute prior stages in the pipeline.

In today's multicore chips, distributing different segments of the image to cores working in parallel can make image processing more efficient. But the way parallel processing is usually done, after each step in the image-processing pipeline, the cores would send the results of their computations back to main memory. Because data transfer is much slower than computation, this can eat up all the performance gains offered by parallelization.

So software engineers try to keep the individual cores busy for as long as possible before they have to ship their results to memory. That means that the cores have to execute several steps in the processing pipeline on their separate chunks of data without aggregating their results. Keeping track of all the dependencies between pixels being processed on separate cores is what makes the code for efficient image processors so complicated. Moreover, the trade-offs between the number of cores, the processing power of the cores, the amount of local memory available to each core, and the time it takes to move data off-core varies from machine to machine, so a program optimized for one device may offer no speed advantages on a different one.

Divide and conquer

Halide doesn't spare the programmer from thinking about how to parallelize efficiently on particular machines, but it splits that problem off from the description of the image-processing algorithms. A Halide program has two sections: one for the algorithms, and one for the processing "schedule." The schedule can specify the size and shape of the image chunks that each core needs to process at each step in the pipeline, and it can specify data dependencies -- for instance, that steps being executed on particular cores will need access to the results of previous steps on different cores. Once the schedule is drawn up, however, Halide handles all the accounting automatically.

A programmer who wants to export a program to a different machine just changes the schedule, not the algorithm description. A programmer who wants to add a new processing step to the pipeline just plugs in a description of the new procedure, without having to modify the existing ones. (A new step in the pipeline will require a corresponding specification in the schedule, however.)

"When you have the idea that you might want to parallelize something a certain way or use stages a certain way, when writing that manually, it's really hard to express that idea correctly," Ragan-Kelley says. "If you have a new optimization idea that you want to apply, chances are you're going to spend three days debugging it because you've broken it in the process. With this, you change one line that expresses that idea, and it synthesizes the correct thing."

Although Halide programs are simpler to write and to read than ordinary image-processing programs, because the scheduling is handled automatically, they still frequently offer performance gains over even the most carefully hand-engineered code. Moreover, Halide code is so easy to modify that programmers could simply experiment with half-baked ideas to see if they improve performance.

"You can just flail around and try different things at random, and you'll often find something really good," Adams says. "Only much later, when you've thought about it very hard, will you figure out why it's good."

Share this story on Facebook, Twitter, and Google:

Other social bookmarking and sharing tools:

Story Source:

The above story is reprinted from materials provided by Massachusetts Institute of Technology, via EurekAlert!, a service of AAAS.

Note: Materials may be edited for content and length. For further information, please contact the source cited above.

Note: If no author is given, the source is cited instead.

Disclaimer: Views expressed in this article do not necessarily reflect those of ScienceDaily or its staff.

View the original article here

Searching genomic data faster: Biologists' capacity for generating genomic data is increasing more rapidly than computing power

ScienceDaily (July 10, 2012) — In 2001, the Human Genome Project and Celera Genomics announced that after 10 years of work at a cost of some $400 million, they had completed a draft sequence of the human genome. Today, sequencing a human genome is something that a single researcher can do in a couple of weeks for less than $10,000.

Since 2002, the rate at which genomes can be sequenced has been doubling every four months or so, whereas computing power doubles only every 18 months. Without the advent of new analytic tools, biologists' ability to generate genomic data will soon outstrip their ability to do anything useful with it.

In the latest issue of Nature Biotechnology, MIT and Harvard University researchers describe a new algorithm that drastically reduces the time it takes to find a particular gene sequence in a database of genomes. Moreover, the more genomes it's searching, the greater the speedup it affords, so its advantages will only compound as more data is generated.

In some sense, this is a data-compression algorithm -- like the one that allows computer users to compress data files into smaller zip files. "You have all this data, and clearly, if you want to store it, what people would naturally do is compress it," says Bonnie Berger, a professor of applied math and computer science at MIT and senior author on the paper. "The problem is that eventually you have to look at it, so you have to decompress it to look at it. But our insight is that if you compress the data in the right way, then you can do your analysis directly on the compressed data. And that increases the speed while maintaining the accuracy of the analyses."

Exploiting redundancy

The researchers' compression scheme exploits the fact that evolution is stingy with good designs. There's a great deal of overlap in the genomes of closely related species, and some overlap even in the genomes of distantly related species: That's why experiments performed on yeast cells can tell us something about human drug reactions.

Berger; her former grad student Michael Baym PhD '09, who's now a visiting scholar in the MIT math department and a postdoc in systems biology at Harvard Medical School; and her current grad student Po-Ru Loh developed a way to mathematically represent the genomes of different species -- or of different individuals within a species -- such that the overlapping data is stored only once. A search of multiple genomes can thus concentrate on their differences, saving time.

"If I want to run a computation on my genome, it takes a certain amount of time," Baym explains. "If I then want to run the same computation on your genome, the fact that we're so similar means that I've already done most of the work."

In experiments on a database of 36 yeast genomes, the researchers compared their algorithm to one called BLAST, for Basic Local Alignment Search Tool, one of the most commonly used genomic-search algorithms in biology. In a search for a particular genetic sequence in only 10 of the yeast genomes, the new algorithm was twice as fast as BLAST; but in a search of all 36 genomes, it was four times as fast. That discrepancy will only increase as genomic databases grow larger, Berger explains.

Matchmaking

The new algorithm would be useful in any application where the central question is, as Baym puts it: "I have a sequence; what is it similar to?" Identifying microbes is one example. The new algorithm could help clinicians determine causes of infections, or it could help biologists characterize "microbiomes," collections of microbes found in animal tissue or particular microenvironments; variations in the human microbiome have been implicated in a range of medical conditions. It could be used to characterize the microbes in particularly fertile or infertile soil, and it could even be used in forensics, to determine the geographical origins of physical evidence by its microbial signatures.

"The problem that they're looking at -- which is, given a sequence, trying to determine what known sequences are similar to it -- is probably the oldest problem in computational biology, and it's perhaps the most commonly asked question in computational biology," says Mona Singh, a professor of computer science at Princeton University and a faculty member at Princeton's Lewis-Sigler Institute for Integrative Genomics. "And the problem, just for that reason, is of central importance."

In the last 10 years, Singh says, biologists have tended to think in terms of "reference genomes" -- genomes, such as the draft human sequence released in 2001, that try to generalize across individuals within a species and even across species. "But as we're getting more and more individuals even within a species, and more very closely related sequenced distinct species, I think we're starting to move away from the idea of a single reference genome," Singh says. "Their approach is really going to shine when you have many closely related organisms."

Berger's group is currently working to extend the technique to information on proteins and RNA sequences, where it could pay even bigger dividends. Now that the human genome has been mapped, the major questions in biology are what genes are active when, and how the proteins they code for interact. Searches of large databases of biological information are crucial to answering both questions.

Share this story on Facebook, Twitter, and Google:

Other social bookmarking and sharing tools:

Story Source:

The above story is reprinted from materials provided by Massachusetts Institute of Technology.

Note: Materials may be edited for content and length. For further information, please contact the source cited above.

Journal Reference:

Po-Ru Loh, Michael Baym, Bonnie Berger. Compressive genomics. Nature Biotechnology, 2012; 30 (7): 627 DOI: 10.1038/nbt.2241

Note: If no author is given, the source is cited instead.

Disclaimer: This article is not intended to provide medical advice, diagnosis or treatment. Views expressed here do not necessarily reflect those of ScienceDaily or its staff.

View the original article here

Searching genomic data faster: Biologists' capacity for generating genomic data is increasing more rapidly than computing power

ScienceDaily (July 10, 2012) — In 2001, the Human Genome Project and Celera Genomics announced that after 10 years of work at a cost of some $400 million, they had completed a draft sequence of the human genome. Today, sequencing a human genome is something that a single researcher can do in a couple of weeks for less than $10,000.

Since 2002, the rate at which genomes can be sequenced has been doubling every four months or so, whereas computing power doubles only every 18 months. Without the advent of new analytic tools, biologists' ability to generate genomic data will soon outstrip their ability to do anything useful with it.

In the latest issue of Nature Biotechnology, MIT and Harvard University researchers describe a new algorithm that drastically reduces the time it takes to find a particular gene sequence in a database of genomes. Moreover, the more genomes it's searching, the greater the speedup it affords, so its advantages will only compound as more data is generated.

In some sense, this is a data-compression algorithm -- like the one that allows computer users to compress data files into smaller zip files. "You have all this data, and clearly, if you want to store it, what people would naturally do is compress it," says Bonnie Berger, a professor of applied math and computer science at MIT and senior author on the paper. "The problem is that eventually you have to look at it, so you have to decompress it to look at it. But our insight is that if you compress the data in the right way, then you can do your analysis directly on the compressed data. And that increases the speed while maintaining the accuracy of the analyses."

Exploiting redundancy

The researchers' compression scheme exploits the fact that evolution is stingy with good designs. There's a great deal of overlap in the genomes of closely related species, and some overlap even in the genomes of distantly related species: That's why experiments performed on yeast cells can tell us something about human drug reactions.

Berger; her former grad student Michael Baym PhD '09, who's now a visiting scholar in the MIT math department and a postdoc in systems biology at Harvard Medical School; and her current grad student Po-Ru Loh developed a way to mathematically represent the genomes of different species -- or of different individuals within a species -- such that the overlapping data is stored only once. A search of multiple genomes can thus concentrate on their differences, saving time.

"If I want to run a computation on my genome, it takes a certain amount of time," Baym explains. "If I then want to run the same computation on your genome, the fact that we're so similar means that I've already done most of the work."

In experiments on a database of 36 yeast genomes, the researchers compared their algorithm to one called BLAST, for Basic Local Alignment Search Tool, one of the most commonly used genomic-search algorithms in biology. In a search for a particular genetic sequence in only 10 of the yeast genomes, the new algorithm was twice as fast as BLAST; but in a search of all 36 genomes, it was four times as fast. That discrepancy will only increase as genomic databases grow larger, Berger explains.

Matchmaking

The new algorithm would be useful in any application where the central question is, as Baym puts it: "I have a sequence; what is it similar to?" Identifying microbes is one example. The new algorithm could help clinicians determine causes of infections, or it could help biologists characterize "microbiomes," collections of microbes found in animal tissue or particular microenvironments; variations in the human microbiome have been implicated in a range of medical conditions. It could be used to characterize the microbes in particularly fertile or infertile soil, and it could even be used in forensics, to determine the geographical origins of physical evidence by its microbial signatures.

"The problem that they're looking at -- which is, given a sequence, trying to determine what known sequences are similar to it -- is probably the oldest problem in computational biology, and it's perhaps the most commonly asked question in computational biology," says Mona Singh, a professor of computer science at Princeton University and a faculty member at Princeton's Lewis-Sigler Institute for Integrative Genomics. "And the problem, just for that reason, is of central importance."

In the last 10 years, Singh says, biologists have tended to think in terms of "reference genomes" -- genomes, such as the draft human sequence released in 2001, that try to generalize across individuals within a species and even across species. "But as we're getting more and more individuals even within a species, and more very closely related sequenced distinct species, I think we're starting to move away from the idea of a single reference genome," Singh says. "Their approach is really going to shine when you have many closely related organisms."

Berger's group is currently working to extend the technique to information on proteins and RNA sequences, where it could pay even bigger dividends. Now that the human genome has been mapped, the major questions in biology are what genes are active when, and how the proteins they code for interact. Searches of large databases of biological information are crucial to answering both questions.

Share this story on Facebook, Twitter, and Google:

Other social bookmarking and sharing tools:

Story Source:

The above story is reprinted from materials provided by Massachusetts Institute of Technology.

Note: Materials may be edited for content and length. For further information, please contact the source cited above.

Journal Reference:

Po-Ru Loh, Michael Baym, Bonnie Berger. Compressive genomics. Nature Biotechnology, 2012; 30 (7): 627 DOI: 10.1038/nbt.2241

Note: If no author is given, the source is cited instead.

Disclaimer: This article is not intended to provide medical advice, diagnosis or treatment. Views expressed here do not necessarily reflect those of ScienceDaily or its staff.

View the original article here