Computers match humans in understanding art

ScienceDaily (Sep. 25, 2012) — Understanding and evaluating art has widely been considered as a task meant for humans, until now. Computer scientists Lior Shamir and Jane Tarakhovsky of Lawrence Technological University in Michigan tackled the question "can machines understand art?" The results were very surprising. In fact, an algorithm has been developed that demonstrates computers are able to "understand" art in a fashion very similar to how art historians perform their analysis, mimicking the perception of expert art critiques.

In the experiment, published in the recent issue of ACM Journal on Computing and Cultural Heritage, the researchers used approximately 1,000 paintings of 34 well-known artists, and let the computer algorithm analyze the similarity between them based solely on the visual content of the paintings, and without any human guidance. Surprisingly, the computer provided a network of similarities between painters that is largely in agreement with the perception of art historians.

The analysis showed that the computer was clearly able to identify the differences between classical realism and modern artistic styles, and automatically separated the painters into two groups, 18 classical painters and 16 modern painters. Inside these two broad groups the computer identified sub-groups of painters that were part of the same artistic movements. For instance, the computer automatically placed the High Renaissance artists Raphael, Leonardo Da Vinci, and Michelangelo very close to each other. The Baroque painters Vermeer, Rubens and Rembrandt were also clustered together by the algorithm, showing that the computer automatically identified that these painters share similar artistic styles.

The automatic computer analysis is in agreement with the view of art historians, who associate these three painters with the Baroque artistic movement. Similarly, the computer algorithm deduced that Gauguin and Cézanne, both considered post-impressionists, have similar artistic styles, and also identified similarities between the styles of Salvador Dali, Max Ernst, and Giorgio de Chirico, all are considered by art historians to be part of the surrealism school of art. Overall, the computer automatically produced an analysis that is in large agreement with the influential links between painters and artistic movements as defined by art historians and critiques.

While the average non-expert can normally make the broad differentiation between modern art and classical realism, they have difficulty telling the difference between closely related schools of art such as Early and High Renaissance or Mannerism and Romanticism. The experiment showed that machines can outperform untrained humans in the analysis of fine art.

The experiment was performed by computing from each painting 4,027 numerical image context descriptors -- numbers that reflect the content of the image such as texture, color and shapes in a quantitative fashion. This allows the computer to reflect very many aspects of the visual content, and use pattern recognition and statistical methods to detect complex patterns of similarities and dissimilarities between the artistic styles and then quantify these similarities.

Share this story on Facebook, Twitter, and Google:

Other social bookmarking and sharing tools:

Story Source:

The above story is reprinted from materials provided by Lawrence Technological University, via EurekAlert!, a service of AAAS.

Note: Materials may be edited for content and length. For further information, please contact the source cited above.

Journal Reference:

Lior Shamir, Jane A. Tarakhovsky. Computer analysis of art. Journal on Computing and Cultural Heritage, 2012; 5 (2): 1 DOI: 10.1145/2307723.2307726

Note: If no author is given, the source is cited instead.

Disclaimer: Views expressed in this article do not necessarily reflect those of ScienceDaily or its staff.

View the original article here

Web-TV: A perfect match?

ScienceDaily (Sep. 3, 2012) — Do you surf the web in front of the TV, or tweet what you are watching? EU-funded researchers are creating technologies that combine web, social media and TV to enhance our experience and interactions across media.

Research shows that consumers watch TV and use the web simultaneously for up to 3.5 hours daily and 42 % of UK adults discussed the programmes they were watching on social networks. Digital providers and broadcasters are always trying to improve entertainment and combining social media, the web and TV into a single user experience is an important step.

NoTube, a European-funded project, brought the digital and broadcasting industries together, along with experts in platform integration, with the aim of linking media together so consumers can watch shows and interact with friends regardless of the devices they use.

'Our prototypes show that the "Web+TV" experiences which most benefit viewers and users will be those using open standards and that work across different hardware, software and service providers. We have tried to develop solutions that give viewers choice and flexibility,' explains Dan Brickley, a researcher at VU University Amsterdam, the Netherlands, one of the lead researchers in the project.

Forging links

The key to NoTube's approach is 'linked data', where information about a viewer -- such as preferences, social networks, contacts and favourite shows -- is stored 'in the cloud'. The data may be held in different databases and formats but it is made accessible by conforming to recognised industry standards for data structure, storage, access and linking.

'The concept of linked data allowed the NoTube team to set reference standards for online publishers. This made it possible, for example, for broadcasters to create personalised news environments and online programme guides, showing users what they most want to see. Moreover, these work across devices and in multiple languages,' says Brickley.

'When NoTube launched, our plan to bring the web and TV closer together via shared data models and content across multiple devices was ambitious and visionary,' Brickley continues. 'Today, the TV industry has caught up, but their cross-platform and personalised services are proprietary. The results and prototypes from NoTube are now more relevant than ever and show the way forward to develop personalised TV applications where the user still controls their data.'

With a vast array of devices and solutions marketed to viewers, it is difficult to achieve a consistent experience when linking online activity and viewing. The NoTube project looked at how this Web+TV combination could work from every angle, developing user interfaces along with underlying technology standards to support interoperability and data linking.

Is linked data secure?

The development of cross-platform solutions was a key focus of the team. 'Hardware engineers at TV companies won't necessarily be skilled at making highly usable programme guide catalogues, or recommendation engines, for example,' explains Brickley. 'As the number of TV channels increases, being able to find and filter the programmes you want will be really useful. We developed a prototype recommendation engine and sharing system which solves this problem and which can be deployed on any media platform.'

Systems using personal data must be secure and respect privacy, which is often a stumbling block for commercial solutions. 'People are often over cautious and misunderstand the risks involved, but they also need to understand how their supposedly anonymous online activities might inadvertently "fingerprint" them. It may take a few more high profile privacy controversies, like the Netflix prize lawsuit or the AOL search logs case, before users adopt healthy privacy habits', said Brickley.

Recognising that people use default settings and fail to guard personal data, the NoTube architecture builds in security to ensure linked data remains secure.

Two media, two screens, many people

NoTube also found ways of linking people viewing TV. Led by BBC R&D, the team developed methods of giving programme recommendations based on social activity and built technologies that make it easier for viewers to discuss and share TV information across their networks, whilst maintaining privacy.

This led to the development of N-screen, a web application which can help small groups decide what to watch. Users share programmes with one another in real time and change the TV channel using drag and drop -- improving the experience of viewers as they watch the same programme, whilst using a second screen to interact with each other.

The project also looked at the possibility of using a smartphone as a TV remote control. 'The key aspect of N-screen or the smartphone remote is that they work by linking different data systems; their functionality is not limited by the type of device or screen used -- giving more choice to consumers,' notes Brickley.

New experiences

The NoTube partners were keen to other functional prototypes, such as the iFanzy service that delivers personalised and contextualised advertising and TV. It uses a range of data, including time of day, device used and viewing preferences, to serve more engaging (and therefore more successful) ads. The system also improves the delivery of audio-visual advertisements by adjusting volumes and automatically selecting the best positioning on the screen.

Another major result is the NoTube TV API which broadcasters can use to build new web-based applications and systems that make TV more interactive and 'do more'. 'The API opens up a lot of what we have developed in the project to broadcasters and media companies so they can build some of our functionality into their own platforms,' Brickley comments.

Looking to the future

'We want the user to be back in the driving seat,' says Brickley. 'NoTube can help people decide what to watch and share, record their preferences, find out more about a programme and have smarter conversations about TV programmes.'

Project partners are promoting results to the technical community; they hope that forward-thinking companies will recognise the potential impact that cross-platform and open source solutions could have. 'Much of our research output and position papers are for a fairly small group of decision-makers in the TV industry and in standards organisations,' Brickley explains. 'But we have received excellent feedback and are involved in various discussions with the W3C standards community.'

The NoTube project received EUR 6.15 million (of total EUR 9.25 million project budget) in research funding under the EU's Seventh Framework Programme (FP7).

Share this story on Facebook, Twitter, and Google:

Other social bookmarking and sharing tools:

Story Source:

The above story is reprinted from materials provided by CORDIS.

Note: Materials may be edited for content and length. For further information, please contact the source cited above.

Note: If no author is given, the source is cited instead.

Disclaimer: Views expressed in this article do not necessarily reflect those of ScienceDaily or its staff.

View the original article here