SOCMINT | Situational Awareness via Twitter

Twitter is by far the platform of greatest interest in terms of event detection. Of all the uses of event detection technology, building situational awareness of rapidly developing and chaotic events – especially emergencies – is perhaps of most clear application to counter-terrorism.

By Jamie Bartlett and Carl Miller

(via www.demos.co.uk) 

Image Attribute: Twitter Art by FavsCo / Creative Commons

Twitter is by far the platform of greatest interest in terms of event detection. Of all the uses of event detection technology, building situational awareness of rapidly developing and chaotic events – especially emergencies – is perhaps of most clear application to counter-terrorism. Emerging events are often reported on Twitter (and often spike shortly thereafter as ‘Twitcidents’) as they occur.

Social media users (especially Twitter users) can play a number of different roles in exchanging information that can detect events. They can generate information about events first-hand. They can request information about events. They can ‘broker’ information by responding to information requests, checking information and adding additional information from other sources and they can propagate information that already exists within the social media stream.

Multimedia content embedded on social media platforms can add useful information – audio, pictures and video – which can help to characterize events. One crucial area of development has been to combine different types of social media information across different platforms. One study used YouTube, Flickr and Facebook, including pictures, user-provided annotations and automatically generated information to detect events and identify their type and scale.

Due to the user generated nature of on social media there is a pervasive concern with the quality and credibility of information being exchanged. Given the immediacy and easy propagation of information on Twitter, plausible misinformation has the potential to spread very quickly, causing a statistically significant change in the text stream. Confirming the validity of the positive system response is a crucial step before any action is to be taken on the basis of that output. A vital requirement of event detection technology is the ability to verify the credibility of information announcing or describing an event. Some promising work has been done to statistically identify first-hand tweets that report a previously unseen story; however it is unclear how that system would perform with the relatively small amounts of data available in an emergency scenario.

Generally speaking, untrue stories tend to be short lived due to some Twitter users acting as information brokers, who actively check and debunk information that they have found to be false or unreliable. One study, for instance, found that false rumours are questioned more on Twitter by other users than true reportage.45 Using topically agnostic features from the tweet stream itself has shown an accuracy of about 85 per cent on the detection of newsworthy events.

One 2010 paper, ‘Twitter under crisis’, asked whether it was possible to determine ‘confirmed truth’ tweets from ‘false rumour’ tweets in the immediate aftermath of the Chilean earthquake. The research found that Twitter did tend toward weeding out falsehoods: 95 per cent of ‘confirmed truth’ tweets, were ‘affirmed’ by users, while only 0.3 per cent were ‘denied’. By contrast, around 50 per cent of false rumour tweets were ‘denied’ by users. Nevertheless, the research may have suffered a number of flaws. It is known, for example, that the mainstream media still drives traffic – and that tweets including URL links tend to be most re-tweeted, suggesting that many users may have simply been following mainstream media sources. Moreover, in emergency response, there tends to be more URL shares (approximately 40 per cent compared to an average of 25 per cent) and fewer ‘conversation threads’.

One important factor, especially important for situational awareness, is the ability to identify the geo-spatial characteristics of an event. Many of the techniques described above to infer the location data of social media content are also used in the field of event detection.

However, the reliability of event detection and situational awareness techniques may be context or even event specific. It appears especially useful in emergency response where a large number of people have a motive to produce accurate information. By contrast, one recent (unpublished) thesis analysed the extent to which useful real time information about English Defence League protests could be gleaned from Twitter.

In the build-up to three demonstrations for which data were collected in 2011, most tweets were negative; and very few were geolocated to the event venue. A very large number of tweets were re-tweets (49 per cent compared to 24 per cent during a control period), and on further analysis, a significant proportion of the re-tweets were negative, inaccurate rumors. Moreover, a very large proportion of tweets (50 per cent) came from a very small number (5 per cent) of – usually negative – commentators.49 The recent crowd-sourced effort to positively ID the suspects in the recent Boston terror attacks on Reddit were also less successful – although it is not clear whether and how information gained through the exercise was of use or value to the police.

This is only one of a number of difficulties relating to the validity and reliability of data sets. There are now, for example, systematic, highly organised operations to create fake reviews, although other researchers are using natural language processing to determine fake reviews from real ones – including verifying an IP address to determine frequency. Of course, at the very large scale, data can be widely skewed by automated information bots. Facebook recently revealed that seven per cent of its overall users are fakes and dupes.

The validity of large scale data sets partly relies not on the fact that every single data point will be taken as accurate, but that when aggregated and combined, large scale data sets can produce valid and robust results – or at least results more robust than any single, even expert, observation. This is the principle that ‘the wisdom of the crowds’ produces more accurate descriptions than any single observer when certain conditions are met: diversity of opinion, independence, decentralization, and aggregation. Social media, as a social network, does not always meet these conditions. One recent study of 140,000 Facebook profiles looked at the first three months of use and found that new members were closely monitoring and adapting to how their friends behaved, suggesting that social learning and social comparison are important influences on behavior. The 2011 London riots were widely discussed – and perhaps partly organised – via social media networks. It does not appear that Twitter was able to ‘dispel’ misinformation quickly. Indeed, rumors spread rapidly, and although some disagreement was found, they were within different, sealed networks.

This article is an excerpt from a technical paper, titled " THE STATE OF THE ART: A LITERATURE REVIEW OF SOCIAL MEDIA INTELLIGENCE CAPABILITIES FOR COUNTER-TERRORISM, published by Demos under Creative Commons license.

Download The Paper - LINK