Social media analysis presents one of the biggest of all big data challenges. The available data is certainly large in terms of size. It’s unstructured text that requires the use of specialized natural language processing techniques and tools to do the analysis, and it never stops coming. Together, those factors can make it a tall order for businesses to make sense of social media data.
But there are good reasons for trying. Knowing what customers and the public in general think of a company and its products can help corporate executives modify business strategies and react to problems before they get out of hand. Social media analytics offers a broader and more realtime alternative to surveying customers and the views of users on Twitter and other social networks are free for the taking. All an organization needs to do is figure out how to tap into and take advantage of the streams of data.
That’s where the problems can start. Simply having access to social media data doesn’t necessarily mean that a business is ready to derive insights from it. Users need to figure out things such as how to score sentiments and how to code algorithms so they can understand nuances like sarcasm and jokes. The notion that data speaks for itself is particularly out of touch when it comes to social media analysis.
“It’s a misnomer that all this data is at our fingertips now and big data has made it available,” said Sarah Biller, president of Capital Market Exchange, a Bostonbased company that analyzes data from social networks and traditional media sources to help investment managers evaluate the price of corporate bonds and assess their bond portfolios. Biller added that organizations need talented individuals inhouse who can normalize and structure all the data being collected so it can be run through analytics engines.
Social media analytics skills in short supply
However, finding that talent is another issue. Nowhere is the shortage of data scientists more acute than in social media analytics. The field is relatively new, and few analytics professionals have the necessary experience and expertise to manage all the complexities of mining social media posts for useful business insights.
Jiri Medlen, a senior text analytics specialist at eBay Inc.’s PayPal subsidiary, said the whole idea of social media analysis is so new that even experienced data analysts often have trouble getting started on it. To make matters worse, he added, there are no commonly accepted best practices on how to quantify the sentiments expressed in text for analysis.
Medlen recommended that companies have specific business goals in mind before jumping into social media analytics. But even then, he said, the ROI from social media sentiment analysis can be questionable. For example, even when he and his team are able to identify negative feelings about PayPal voiced on social media that may lead a PayPal member to close his or her account, it may be too late to change that person’s mind. Knowing a customer’s sentiment doesn’t necessarily translate to meaningful action.
“The biggest question is what to do with this kind of data,” Medlen said. “How is this going to impact the bottom line of the company? We still have to answer the question of value.”
Technical difficulties with social media data
And while social media data can deepen analytical models and findings, it can also add a new layer of complexity that poses technical challenges that some businesses may not be ready to handle. Speaking at the 2014 RapidMiner World user conference in Boston, Usama Fayyad, chief data officer at Barclays PLC and former CDO at Yahoo Inc., said the rapid streaming and unstructured nature of social data makes integrating it into analytics systems a big headache.
Fayyad said he and his team at Yahoo used to think the standard 25 characteristics that the Internet company collected on every user qualified as big data really big data. That information included things like age, demographics and search history, data used mainly to develop profiles of Yahoo users for targeted marketing. But then the analysts started to look at how they might be able to use data from the Facebook, Twitter and LinkedIn accounts of users and that truly introduced them to the challenges of working with big data, Fayyad said.
Early tests of social media analytics applications showed promise, he added, but dealing with the complexity of the unstructured data proved difficult. “We really don’t know how to deal with variety,” Fayyad said. “You can enhance any data set with all this and it makes it better. But it’s a monster.”
Ed Burns is site editor of SearchBusinessAnalytics. Email him at eburns@techtarget.com and follow him on Twitter: @EdBurnsTT.
Author: Ed Burns