I’ve been exploring paid social media monitoring systems as part of my role at FirstGiving. Initially, I was pretty excited about being able to monitor social media sentiment. Measuring aggregate sentiment over time seemed obvious to me. However, it became clear that none of the tools monitored sentiment well. In fact, they were all terrible at returning real online user sentiment. Neutral tweets were marked as negative or positive, positive tweets were marked as negative, and so forth. After researching sentiment analysis, I was not surprised to read that automated sentiment analysis is slightly less accurate than flipping a coin to determine whether or not brand mentions are positive or negative. (I tried to tweak the returns by manually changing the sentiments to influence the automatic marking, but that had no effect on accuracy.) I had been thinking that measuring sentiment analysis was the key to getting real ROI. I was completely wrong.
Measuring social media sentiment is the measuring the wrong ROI.
Sentiment analysis and measurement at companies is often driven by the need to prove the ROI (return on investment) of social media. Pressured to prove the value of social media involvement, the person in charge of social media will be tempted to prove his/her value by highlighting the percent change in positive online mentions. I would argue that measuring the change in sentiment is the wrong ROI to measure.
The right ROI measurement should be increased engagement and interactions.
Positive brand sentiment is nice. Does it move people to action? US internet users are likely to share information about your company and proactively recommend that someone make a purchase online, so engaging with stakeholders online supports that behavior. You really want to engage in such a way as to add value and create brand loyalty. Taking action as a result of social media is the ultimate ROI.
With the right engagement activity, loyal clients and consumers should naturally take the actions that you want them to take. Begin by measuring the level of online engagement with your organization, and make your plan based on that information. If your activists/clients/consumers are not talking to you at all online, that’s the engagement equivalent of negative sentiment. If they mention you in passing, that’s the engagement equivalent of neutral sentiment.
Let’s turn the ideal ROI of increased positive online sentiment on its head, and replace it with measuring increased positive online engagement with the organization. I wrote a similar statement in the comments of Shel Israel’s wonderful blog post describing his mixed emotions about sentiment analysis. Tal Wolgroch replied in the comments that “sentiment analysis is not about traditional ROI anymore. Those who try to measure it this way are playing a new game with an old rulebook.”
Stop trying to measure sentiment. Worry about your lack of engagement.
Pingback: How: Measuring Sentiment « Katy Leuschner's Blog()
Pingback: Acerca del Retorno de la Inversion (ROI) en las Redes Sociales – LEC Temporal()