Home > Data Services, HANA, Predictive Analytics > Sharknado Social Media Analysis with SAP HANA and Predictive Analysis

Sharknado Social Media Analysis with SAP HANA and Predictive Analysis


Mining Social Media data for customer feedback is perhaps one of the greatest untapped opportunities for customer analysis in many organizations today. Social Media data is freely available and allows organizations to personally identify and interact directly with customers to resolve any potential dissatisfaction. In today’s blog post, I’ll discuss using SAP Data Services, SAP HANA, and SAP Predictive Analysis to collect, process, visualize, and analyze social media data to the recent social media phenomenon Sharknadosharknado

Collecting Social Media Data with SAP Data Services 

While I’ll be focusing primarily on the analysis of social media data in this blog post, social media data can be collected from any source with an open API by using Python scripting within a User-Defined Transform. In this example, I’ve collected Twitter data using the basic outline provided by SAP in the Data Services Text Data Processing Blueprints available on the SAP Community Network, updated it for the REST version 1.1 Twitter API.  This process consists of 2 dataflows, the first tracks search terms and constructs (Get_Search_Tasks transform) and executes (Search_Twitter transform) a Twitter search query to store the data pictured below. In addition to the raw text of the tweet, some metadata is available, including user name, time, and location information (if the user has made it publicly available).

Picture1

Once the raw tweet data has been collected, I can use either the Text Data Processing transform in SAP Data Services or the Voice of Customer text analysis process in SAP HANA. While both processes give the same result, SAP Data Services is also able to perform preliminary summarization and transformations on the parsed data within the same dataflow. In this case, I will run text analysis in SAP HANA by running the command below in SAP HANA Studio.

Create FullText Index “VOC” On ()

TEXT ANALYSIS ON

CONFIGURATION ‘EXTRACTION_CORE_VOICEOFCUSTOMER’;

This results in a table called $TA_VOC in the same schema as the source table, as shown below.

sharknado2

To download full PDF and Continue Reading…

Hilary BlissAbout Hillary Bliss
Hillary is the Analytics Practice Lead at Decision First Technologies, and specializes in data warehouse design, ETL development, statistical analysis, and predictive modeling. She works with clients and vendors to integrate business analysis and predictive modeling solutions into the organizational data warehouse and business intelligence environments based on their specific operational and strategic business needs. She has a master’s degree in statistics and an MBA from Georgia Tech.

  1. July 30, 2013 at 10:15 am

    Reblogged this on All Things BOBJ BI Blog.

  2. Zenith
    September 13, 2013 at 12:59 am

    Hi,
    I am unable to find the webinar session on “Twitter data collection and social media sentiment analysis by Nicholas Hohman”, as mentioned in the pdf.Can you please send me the webinar link to zenith.reddy@tcs.com

  3. September 20, 2013 at 12:18 pm

    Great example of a Twitter stream analysis. I can think of lot’s of potential applications of this type of work.

  4. December 13, 2014 at 3:49 pm

    Good response in return of this matter with genuine arguments and describing everything about that.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: