As virtual private assistants like Amazon’s Alexa become smarter, more popular and more useful, why not give it them tools to recognize emotions? That seems to be the thinking behind Israel-based Beyond Verbal, which makes voice recognition software to analyze human emotion and health indicators, releasing a cloud-based API engine to integrate with virtual private assistants (VPAs).
The intention for Beyond Verbal’s latest project is to make one’s connected home more sensitive, so to speak, rather than a sounding board that simply takes request after request. Operating on the knowledge that vocal intonations represent 35 to 40 percent of emotions conveyed in human communication, Beyond Verbal set out to eliminate the emotional blind spots of VPAs. The company’s Emotional Analytics technology takes 10 seconds of raw voice input and analyzes it for mood and attitude, then factors in that information with the user’s request. For example, if someone asks their VPA to play music, the Beyond Verbal-enabled bot will take note of the person’s emotional state – say, stressed out – and respond with a list of calming music choices.
“Today’s digital world is rapidly transforming the way we interact with our technology and each other. Virtual private assistants have begun to take on a personalized experience,” Beyond Verbal CEO Yuval Mor said in a statement. “We are very excited for this next step in fusing together the breakthrough technology of AI and Beyond Verbal’s Emotions Analytics, providing unique insight into personalized tech and remote monitoring.”
Beyond Verbal, which was founded in 2012, has made a name for itself with its straightforward consumer technology built on years of hard science research. The company is actually the product of several research projects spanning 21 years, and the company has collected more than 2.5 million emotion-tagged voices in more than 40 languages to analyze human emotions. It launched its Beyond Wellness API in 2014, which turns any smartphone or mic-equipped wearable device into an emotional wellbeing sensor using technology that doesn’t consider the actual content or context of spoken word, but instead studies intonation in the voice. The company has two free, consumer-facing apps, Moodie and Empath, and one for clinicians called Beyond Clinic.
Just getting VPAs accustomed to their owner’s emotions is the first step, Mor said. Eventually, they want to layer in insights from their ongoing research on vocal biomarkers of health conditions, which has recently ramped up. The company has been working with the Mayo Clinic, Scripps, Haddassah Medical Center in Jerusalem and Tel Aviv University and others to expand research into health-indicating vocal intonations. In September, Beyond Verbal launched a research platform to identify physiological markers through the voice that may indicate various health-related issues called the mHealth Research Platform, the company will enable collaboration with other institutions, medical centers and commercial organizations.
Beyond Verbal is one of several health tech companies making moves to leverage VPAs as of late. In February, Lenovo Health and Orbita launched a voice-enabled home health assistant. As of last March, people who use Amazon’s VPA can launch the WebMD skill on any Alexa-enabled device (such as the Echo, Echo Dot and Amazon Fire TV) and ask a question about a range of health-related topics including conditions, medication, tests and treatments. Alexa will respond with WebMD-sourced answers in easy-to-understand language.
In April, Merck and Amazon partnered up to launch a developer competition to incent startups and developers to create apps that harness Amazon’s Alexa technologies for people with type 2 diabetes. Boston Children's Hospital officially launched its partnership with Amazon Echo that same month, giving the voice-powered home appliance a new "skill" that will allow it to give simple health advice to parents about their children's' fever and medication dosing. Called KidsMD, the Alexa app is just the first step in a plan to bring Boston Children's medical knowledge to the consumer space, according to Chief Innovation Officer John Brownstein.
But all those are built on spoken requests, and Beyond Verbal wants to go deeper by actually recognizing health conditions rather than waiting for their owner to ask about it by name.
“In the not so far future, our aim is to add vocal biomarker analysis to our feature set, enabling Virtual Private Assistants to analyze your voice for specific health conditions,” Mor said in a statement.
