One of last holiday season’s most popular gifts was home assistants, like Amazon’s Alexa and the Google Home device. In January, someone having a bit of fun decided to live-stream two Google Home assistants talking to one another—named Vladimir and Estragon after characters in the play Waiting for Godot—and the exchange was pretty funny.
As technology becomes more interactive, there is the potential that we will mistake this type interaction for understanding. Although it feels like the devices are conversing, it’s probably a more accurate observation to suggest that they are simply responding to prompts provided by each device in a loop. Chatbots, which are becoming more widely used to respond to service inquiries, are similarly limited. This is fine, in both the case of home assistants and chatbots, there is a service goal that the devices are programed to deliver.
Real social interaction online occurs between human beings, using their messy, complex languages and full of sarcasm, humor, and cultural context. If you are mining social interactions to uncover insights into your marketing or public relations work, you’ll need humans reviewing the content.
As good as natural language processing is getting—and it is getting better—human review is still critical if accuracy is important.
That one friend who…
We all seem to have that one friend who has a sense of humor that is so dry you spend half of the time you’re with this individual trying to figure out if he or she is kidding around or being serious. There are also the sarcasm delivery experts, where context is key to determining what they are attempting to convey. It’s hard enough to parse some of the statements these individuals make when they are sitting next to you—and these are people you know personally. It can be even more challenging when trying to discern the sentiment behind a statement when reviewing the written word—situations like this are exactly how internet adages like Poe’s Law come into being.
Expecting automation to sift through comments and feedback and correctly deconstruct dry humor and sarcasm isn’t realistic. It takes a human to view the content and back-and-forth in the discussion thread to analyze the probable intent behind a statement.
Enter stage left…
Dramatic content presents a similar burden to automated analysis. I have a number of friends who are currently parenting adolescents. Several have offered to sell their offspring after having particularly challenging exchanges with them. These offers aren’t limited to the parents of human children—one friend stated she was selling her dog after he managed to open the freezer and consumed several steaks. The one thing these people all have in common is that they weren’t actually interested in selling their children or pets via Facebook; they were expressing frustration with behavior. People know this intuitively—a computer does not. Drama, hyperbole, and exaggerated claims are statements driven by emotion that humans can process accurately but an algorithm cannot.
Divided by a common language…
American English and British English can vary considerably, particularly in idiomatic expressions. “Bob’s your uncle,” “gone pear-shaped,” “sound as a pound,” and “sticky wicket” are not phrases commonly heard in the U.S. Conversely, “bought the farm,” “shoot the breeze,” and “ride shotgun” are Americanisms. The British say “trousers”— “pants” to them means underwear.
These examples barely scratch the surface of language differences just within English. Now imagine trying to translate idiomatic expressions from other languages and cultures (check out BuzzFeed for some seriously “lost in translation” signage errors: warning, some are rather colorful).
If you are examining social content for a multinational company, automated translation apps aren’t going to be useful or accurate. Exchanges on social channels are informal discussions that are going to be full of slang, idioms, and culturally local context. Analyzing this type of content requires a fluent speaker if there’s any hope of translating the meaning accurately. Even better than a fluent speaker is someone who is on the ground in the country, as even fluent speakers might miss local or regional references or differences in dialect.
Human analysis of social content offers more than simply a gut-check on accuracy. It provides a much-needed layer of understanding to the informal exchanges that dominate social media. The informal and authentic nature of discussion that is prevalent on social media is exactly where it provides value to companies collecting and analyzing the content—unvarnished opinions carry weight. However, if the information collected isn’t viewed with precision, the value of the content declines. If a company is going to spend the money to monitor social platforms, it makes financial sense to have that content analyzed by humans, so that any business actions based on social media insights are accurately informed.