United Nations' Global Pulse initiative partners with social data provider DataSift to analyze worldwide opinions on vaccinations, poverty, and other humanitarian topics.
United Nations data scientists working in New York; Jakarta, Indonesia; and Kampala, Uganda, will use data aggregation and filtering tools from DataSift to study attitudes of people from around the world, the organizations announced Tuesday.
Researchers from Global Pulse, a UN innovation lab that uses big data to foster sustainable development and humanitarian efforts, will use the DataSift platform for social data analysis across a variety of projects, including those pertaining to poverty, hunger, vaccinations, and other pressing issues, according to Jason Rose, DataSift's senior VP of marketing.
"People often think of social data analysis as useful primarily for monitoring brand sentiment and doing customer service," Rose said in a phone interview with InformationWeek.
[Are you doing the right things with your big data? See Healthcare Data Analytics Gone Wrong.]
DataSift hopes the UN Global Pulse partnership will change that perception. The company processes 2 billion social interactions per day across a variety of global platforms, including Twitter, Tumbler, and WordPress.
"Our engine is designed to take and filter those down to [several] thousand, or whatever the number is that matters to a particular topic area or organization," Rose said.
Global Pulse will use DataSift in a variety of ways. One project will examine attitudes toward immunization in order to improve the success of vaccination campaigns and to prevent the spread of disease. Another will explore how families cope when faced with sudden spikes in food prices.
The streaming-data tool helps UN data scientists quickly discover and analyze emerging trends.
"This has enabled us to create new techniques and methods that give development practitioners the valuable insights needed to improve the well-being of communities all over the world," said Global Pulse director Robert Kirkpatrick in a statement.
One potential use is early detection of disease outbreaks, as well as identifying issues and attitudes toward immunization, said Rose.
Using online data to gauge real-world problems can be tricky, however. When Google launched its Flu Trends website in 2008, for instance, it claimed the service -- which analyzes aggregated search queries -- could detect disease outbreaks faster than global health agencies. But during the 2012 to 2013 flu season, Google Flu Trends overestimated the incidence of flu in the US -- a result of its algorithms not being tuned to account for "heightened media coverage" of flu outbreaks, Google said.
But "that's not the work DataSift is doing," said Rose. Rather, the DataSift software allows the Global Pulse data scientists to apply their own algorithms.
"They are maintaining, updating, tweaking, and testing their algorithms based on the information we're feeding them," he added. "We also have a historic data store. For many of our data sources, you can go back three or four years and back-test all of your algorithms against known outbreaks and events that have happened in the past, and calibrate your algorithms that way as well."
Rose also noted that Twitter isn't the only source of DataSift's social data stockpile. "WordPress is another key source," he said. "Twenty-two percent of the world's websites are running on WordPress."
The UN effort shows the potential of social data analysis, he added.
"We are scratching the surface of the applicability of social data, and I applaud nonprofits like the Global Pulse initiative that are going out there and applying it in new and innovative ways that are doing good in the world," Rose said.
