Tag Archives: automated sentiment analysis

Measurement Tools – helping us understand or drowning us with data?

Featured

Posted on by
Reply

It remains a curious irony that in a world never richer in data than it is today we are sometimes poorer in information terms than we were, say, a decade ago. This is certainly true for anyone trying to manage a brand or company reputation where the ability to collect millions of items about brands, products and customer insight is relatively easy – but the volumes and complexity of the data has made getting any useful information out of it harder than ever. Many tool vendors in this space don’t help matters – like people who confuse precision with accuracy. They confuse data with information, some even confusing data with knowledge or even intelligence – not good at all.

To what extent can metrification help?

Before we go anywhere near metrics and dashboards there are a few basics to cover. The foundations of any successful system are going to be: access, accuracy and context. It’s the last one which lets most systems down. They struggle to assign arbitrary values based on nonsensical assumptions that ‘absolute accuracy’ is a valid concept. On its own accuracy is impossible without the context within which to frame it. And therein lies the problem: how can an automated system ‘know’ what someone is looking for? To quote a long-time client: “…finding 20,000 articles about my company is easy – finding the 20 I need to know about right now isn’t”.

Two things to note here:

  1. The ‘right now’ introduces a fourth element which though not part of the system solution is nonetheless a very valid concern. With results timing is everything. Want to know last week’s lottery winning numbers? Thought not.

Now imagine having to track every lottery combination manually before the draw. Impossible? You bet it is.

  1. While in the traditional media space human analysis may at least be theoretically possible, in the social media space the volume would overwhelm any attempt to analyse this data within an actionable time frame.

That’s not to say there isn’t a strong role for human analysis but let’s not turn them into glorified data entry clerks which brings us to what’s now possible with real-time automation.

Learn to love your automated analyser and it will love you back!

Enter the new generation of analyser which combines natural language (so you don’t need to learn all manner of combinations of AND, OR and NOT) when asking questions with the ability to use your specific context as a guide to filtering and refining results. Science fiction? Not anymore.

The beauty of using such advanced techniques is that they don’t require you to understand much, if anything, about how they work. Put simply, if you want the system to tell you how well a new product ‘X’ is being received, you can type in “what are reactions to ‘X’?”. Then leave the analyser to figure out all the variations it needs to answer the question from your perspective. Getting perspective right is the key to accurate sentiment. Of course, you (with the assistance of the system provider) have to define what it is you want to know at the start. This is typically something that takes a few hours and then it’s done. The smartest of this new generation have some feedback or learning capabilities to help the system evolve as your company changes and improve its guesses and scores for context.

Compare that to the current crop of ‘staples’ in the industry. Some actually make a point of showing how complicated their processes are. For example, I’ve seen one query which filled a whole screen just to make sure that if you are interested in Apple smart phones your query didn’t bombard you with cookery tips or news about where to buy an orchard.

A side benefit of the new method is that it may actually help you to learn how to better engage with your customers by looking at how they express their opinions. And because it’s quick to try new ideas you can afford to try different approaches without having to worry about wasting time if your first ideas are wrong.

If the worst aspect of an analyser is that it forces you to think harder about your brand, business or communications strategy then I’d call that a success.

Keith

An introduction to GlideIntelligence

Posted on by
Reply

Ever since I started Glide Technologies in 2003, clients have been telling me that the methods available for effectively measuring performance in the media are insufficient. Complaints have typically been that services are expensive, slow, highly subjective and inaccurate. My belief has always been that technology will deliver a better way and now, seven years on, the technology is finally here to realise the dream.

In the last two years, the media landscape has changed dramatically and it’s changed forever. Companies are fast realising that what worked yesterday, simply doesn’t work today. Leading brands are changing the way they measure and for good reason. Gone are the days where a company’s reputation could be managed via a controllable set of key media channels. In today’s socially-networked world, reputations are exposed across hundreds or thousands of media outlets, including blogs, social media websites, broadcast and traditional media. Furthermore, the simultaneous fragmentation and democratisation of media means that media is frequently global. There is, therefore, an urgent need to watch and listen across very large volumes of information. Quickly getting a view on what’s happening is essential. The days of rear-view-mirror-reporting are gone because, no matter how good the report is, it’s of very limited decision-making value if it’s received after the event in the typical month-end or quarter-end format.

We began developing GlideIntelligence two years ago (after a year in planning and consultation) and were fortunate enough to attract Keith Woods-Holder to lead the project. Ex Research Director at Saatchis, Keith spent over ten years developing automated sentiment analysis models for the likes of Dell (working directly with Michael Dell himself), Sony, IBM, Kodak and Barclays. The plan was simple – take Keith’s twenty years experience in the sector, give him a highly-talented development team and all the latest technologies, and create a software-as-a-service  semantic measurement platform, GlideIntelligence. The results have surpassed even our greatest expectations and we now have a powerful, fourth generation model which overcomes many of the criticisms that have been levelled at past attempts at automated sentiment analysis while creating real competitive advantage.

GlideIntelligence is a contextual measurement solution. Unlike previous generations of analyser, it does not use dictionaries but, instead, uses grammar and context. This means that it can learn new language quickly (including slang and non-English phraseology common in social media) and will not be constantly out-of-date with language nuance. This also allows us to overcome many of the legacy criticisms of automated sentiment, for example allowing it to handle sarcasm and irony.

GlideIntelligence was developed in conjunction with a select number of Glide’s large global brands. This ensured we did not develop in a vacuum, while keeping us close to the issues and priorities of business. A nine month beta programme has helped iron out any creases, while readying the product for commercial launch.

We felt it was essential to deliver complete transparency within our product. There are a lot of automated measurement tools on the market and a common complaint is not just that they don’t do what it says on the tin, but that there is a great deal of opacity about the methodologies used. We call this the ‘black box’ factor. Namely, data is fed in one end and charts out the other, but it is difficult or impossible to know how the sentiment was deduced nor the charts constructed. GlideIntelligence overcomes this restriction with in-built transparency. The way sentiment has been deduced is completely visible. We’re confident enough in our accuracy rates to let customers see this for themselves.

In today’s highly-competitive, connected world, it’s never been more important to be able to see the whole landscape, including competitor and industry movements. This is why we built in multi-perspective analysis. This gives the corporation the ability to benchmark effectively and quickly spot competitor strengths and weaknesses. GlideIntelligence creates the ability to see what an article means not just for you or your product, but for any number of companies, brands or products.

Another common complaint from the industry is the amount of time and expertise required to work with modern measurement platforms. Common to our founding value of making software easy to use, with GlideIntelligence we made sure that insightful charts and reports could be built with a few clicks of the button. Furthermore, a MyReports feature allows favourite charts to be saved for one-click access. A powerful alerting feature makes it easy to push important news stories to users based upon their needs. For example, it is possible to receive an instant alert about all negative coverage concerning the CEO.

Real-time analysis will fast become the norm and organisations without it will not be able to manage and respond in time to important market developments. Media will continue to diversify, fragment and overlap and organisations will increasingly need to think of media as global and not local because the internet does not discriminate based upon location. These changes create new challenges, in turn forcing us all to look for new, more effective ways to measure the corporate reputation. GlideIntelligence is part of that exploration.

Sam Phillips, Glide’s Founder and CEO

The Challenges of Measuring the Engaged Web: The Engaged Web Part III

Posted on by
Reply

This is the third and final part in my Engaged Web series of blog posts.  If you missed the previous posts, you can access them using the links below;

Part I: The Engaged Web, Part II: Speed is the New Currency

3 key challenges

There are 3 key challenges which organizations face when measuring engaged two-way media:

  1. Sentiment is fluid and can change rapidly online. What begins as positive can change to negative and even back to positive. In other words, it evolves. It’s crucial, therefore, to look at trends and movement rather than just volume.
  2. Defining what is positive or negative is based upon your point of view. What good for one organisation is not necessarily good for another. Perspective is paramount.
  3. The engaged web has its own language. The syntax used on Facebook and Twitter is very different to that of conventional prose. Think hash tags, emoticons etc.

In my view, the measurement industry needs to move from looking at ‘what has happened’ to ‘why it has happened’. But, as we see above, there remain some real challenges.  Effectively measuring reputation requires measuring all reputational influence.  We must measure traditional media (newspapers, magazines etc) alongside the Facebooks and Twitters of the world. If we don’t, we will fail to understand trends and patterns and establish the true connections.

The Case for Automated Sentiment Analysis

So how does an organisation know at the right time what is being said about it across tens or hundreds of thousands of media channels? It simply cannot be achieved quickly enough (let alone cost-effectively) with human evaluation methodologies. This is why more and more companies are turning to automated sentiment analysis.

Sentiment analysis engines have traditionally used a dictionary-based approach to measuring and identifying sentiment.  This method needs a dictionary of at least 250,000 words to be anywhere near effective.  It also means that it must be constantly updated with new words if it is to stay on top of the latest linguistic nuances.

However, superior tools are now being developed which operate around rule-based methodologies that do not use dictionaries, but instead analyze grammar and context.  This allows them to have a far greater level of language independence plus the ability to cope with slang and the other syntax challenges previously mentioned.  Tools which use this approach also have the ability to self learn and automatically adapt to language change as it happens which is critical when measuring the constantly-evolving engaged web.

Over the years, automated sentiment analysis has had a few false horizons and there is, quite justifiably, cynicism from some as to its efficacy. But this is now being cracked. Huge gains are being made in accuracy, speed and usability.  As this develops, the world will become divided into those who great reputation management software and those who don’t.  There will be clear winners and losers. The winners will be using sentiment analysis platforms to elevate the human role to high level analysis and decision making, while those who don’t will be left drowning in thousands of pages of posts and tweets, wondering where it all went wrong.

Sam