Xbox One, Kinect 2.0 and the future of health technology

By: admin | May 26, 2013        

Tags: | | |  |

Marcelo Calbucci EverymoveMarcelo Calbucci is the Co-founder & CTO of EveryMove and a former development manager at Microsoft

Here is a bold statement: No other device, with the exception of the smartphone, will do more for the evolution of health technology than the Kinect 2.0 on the new Xbox One.

In case you are not a gamer or didn’t follow the news, the new Kinect comes packed with lots of high-end gear like a high-resolution digital camera, an infrared camera and several high-fidelity microphones. But those are just sensors, the magic happens on the software. Microsoft released the first Kinect just three years ago, but they’ve been working on the software for more than a decade, and they’ve built a platform, which means new apps (not only games) will be able to tap into an amazing source of information on our body.

The first category of games to benefit from these high-end sensors is fitness games, which are already leveraging the previous version of Kinect, and also made Nintendo’s Wii Fit hyper popular. Next generation fitness games will be able not only to tell you how to make a downward dog or a squat, but the games will be able to tell you to keep your back straight, or move your left foot six inches to the right. Now we can start imagining a RunKeeper or MapMyRun app for Xbox that focuses on training for a run that beautifully integrates with your training plan.

The fitness apps will be the first to come, and over the next one to two years we will see a flood of them. However, the apps likely to make the biggest difference in health will need another three to five years of development before they hit the market, but they will be nothing short of Sci-Fi turned reality.

Kinect-enabled Medical Tricorder

In case you haven’t seen yet, check out this video of the Xbox Kinect detecting a person’s heart rate just by changes in the colors on their skin. This technology was invented by a group of researchers at MIT, and Xbox Kinect is a great way to showcase the applicability of it. Now, I’ve been immersed in the fitness space for the last two years, and one thing we know is the importance of measuring heart rate during physical activity to understand the longterm impact of that activity on an individual’s health. Currently, heart rate monitor is not commonplace, but the Xbox Kinect can change that.

The new Kinect is precise enough that you can measure not only the heart rate of an individual, but the Heart Rate Variability (HRV), which can lead to the diagnosis of many conditions, including stress, depression and many other mental and physical conditions. How exciting is it to imagine standing in front of your Xbox to be given a health report?

And there are scenarios that could exist with this new Kinect that simply don’t exist today. Here is an idea: How about Kinect takes a picture of your body and it’s able to compare to previous pictures and detect new moles, skin conditions or even skin cancer? Absolutely possible! The technology will be there in the next couple of years. There are privacy, ethical and legal concerns, and we’ll need to work through those, but the tech will work.

Back to fitness, we might be able to get an approximate VO2 max for individuals by measuring their movements and heart rate change during an activity and compare that to other data collections over the last few weeks. Athletes could benefit tremendously from this, without the need to pay hundreds or thousands of dollars for high-end exams. It won’t be as accurate as an EKG and a mask measuring precisely your O2 and CO2 input and output, but, hey, it’ll be accessible by many.

Telemedicine and other future apps

I can imagine a whole new class of apps that collect, analyze and provide all kinds of interesting insights into an individual fitness and health. Here are just a few top of mind ideas:

  • Telemedicine: We have all seen the videos from the 90s from IBM, Intel and Cisco touting telemedicine. But those were bulky, hugely expensive solutions. The Kinect will be able to give your doctor a true peek at your body, with the ability to zoom-in and out, get your heart rate and blood pressure (maybe), see your HRV without plugging you into a single machine and get a report from Kinect of how your weight, joints movement, skin and voice has changed over the last weeks or months.
  • Vocal cords & lung conditions: There is an iPhone app from researchers at the University of Washington Medicine and Seattle Children’s Hospital today that can measure the health of your lung by you breathing into the microphone of an iPhone. We’ll have that on Kinect. Not only that, but the Xbox can ask you to breathe or speak to detect conditions on the spot, like throat inflammation. Maybe it will be able to plot your voice patterns over time and detect that over the last three months you have been consistently changing your pitch. Time to see a doctor!
  • Personal trainers: With Skype plus Kinect plus detailed motion detection, a personal trainer can not only give you guidance from 800 miles away, it can also use the augmented information from Kinect to notice that your ankles are not on a 90-degree angle with your foot when doing a lunge, put a red dot on your foot and let the trainer communicate to you what’s the right form.
  • Face change: With advanced software, the Kinect could detect any changes to your facial expression. Detect different coloration of your eyes, skin or tongue (yes, it will be weird to say “ahh” 12-inches from the TV). You’ll be able to detect inflammation on the lymph nodes on your neck, or even a millimeter of change on the distance between the left corner of your lip and your left eye. All that in just a few seconds.
  • True METs: The compendium of physical activities is a great source if you want to know how many METs running on a treadmill consumes, but lots of activities have approximated METs, and it’s usually not exactly what you are doing, it’s what someone in some lab was doing. By measuring the detailed movements of your body and your heart rate, we’ll figure out new and interesting ways to compute METs and be able to deliver true value for that physical activity. And imagine what happens when you correlate the data of 100,000 people who did yoga and their heart rate changes, breaths per minute, and motion? It’s hard to comprehend the value this data will bring.

It won’t be all sunshine and lollipops. As I mentioned before, there are numerous issues to be addressed, from the software that needs to be built, to the social acceptance of this radical shift in health technology. There are privacy, legal, ethical and security issues.

The fact is this evolution in health tech will happen, and there is no stopping. It will take a full five to seven years for most of what I describe to become reality, and we should embrace it. The upside for the individuals and society far outweigh the dangers.

About the author: Marcelo Calbucci is the Co-founder & CTO of EveryMove, a startup in Seattle working with health plans, gyms, brands and employers by rewarding individuals for their healthy lifestyle. He is also a former development manager at Microsoft. You can follow him on Twitter @calbucci

  • artofrx

    are u really dat stupid..??god!!heart rate…seriously..dude i am a doc and preparing for residency…heart rate seriously…anyone can check their heart rate by placing 3 fingers on the palmar wrist aspect of the other hand…so do u expect people to but the shitty kkinect just to check theis heart rate or the pulse which they can do by using the 2 hands…and regarding the diagnosis of disease…there are other symptoms that would appear way before significant changes in heart rate…heard about the anginal chest pain…so dont need to feed bullshit here…technology development going in the wrong direction….

  • Heretic

    It is not about heart beat measurement my friend. Technology takes different shapes and hues. The end applications can be amazing, only aiding the doctors like us and human beings in general.

  • James Cox

    And all you need to do is give up your privacy to have an always on, internet connected spycam in your house that feeds back data to Microsoft and third parties. Not to mention hackers. Get real, the whole biometric feedback is for gauging your interest in the ads or TV shows you are watching. This is no Medical miracle. It just reads your pulse. Any person can do that. Spin Spin Spin…… Nope.

  • TheOtherTurnipTaliban

    If you are a doctor then I am Jimmy Carter.

  • Tom Chikoore

    With the exception of the IR heart rate monitors, at Motion Nexus ( ) we have the platform that combines most of the ideas/technology that you are writing about. We have a platform that aggregate, correlates and interprets data from fitness/health device (pedometers, heart rate monitors, BP monitors, digital scales ), motion sensors (Kinect, Primesense) and other data sources. Here is is some of the work we have done for the the Kinect an how it is being used as relating to:

    1) Telehealth (At home PT):
    2) Social Fitness (Exer-Gaming):

    Our technology has been on the market for a year.Despite the shortcomings of the current version of the Kinect, we have made it work on PCs and Mac without the need of an XBOX.

    Most of the XBOX One Kinect “future” announced last week is already here today.

  • Jack D. Wilson

    It remains to be seen if your “bold statement” is realized. Right now fitness monitors like FitBit are leading the pack. I want to see some real “value added” applications on XBox One before I drop the dollars I have heard it will cost (reported as $600- in the UK). Something like a Zumba coaching application might tempt me. Right now we have lots more PR than substance and in my way of thinking I do not make purchase decisions based on PR (either from MS or one of their avatars).

  • goolia


  • me

    it says that they can detect your heart rate through the detection of your skin color…but what if you have really dark skin color? lol does that mean it might not read my heart rate and it will think I’m dead?

  • Bill E Weaver

    Imagine a game like LA Noire, the kinect senses you are lying about being undercover due to natural reactions and the game plays out differently. The technology is there, it’s up to the publishers to make innovation now.

  • John

    What a hell is “left lip”?

  • rabbitlug

    The narrowness of your thinking is quite breathtaking.

    Explain how this (essentially dumb) device reports back to MS? That would be done at the application / OS level of the system, not the camera, and if it’s done at the app / OS level, then it can be changed.

    “Third Parties”…. what, like doctors and hospital staff. That would be absolutely terrible wouldn’t it?

    Hackers: OMG, the 1337 h4X0rz!! RUN!!! Your medical records are almost certainly on a hospital network right now, your baking details are on some internet connected server somewhere and your personal data is all over your HDD. The Internet is not 100% secure… but you use it don’t you?

    Hospital networks are run over private leased lines for security, they don’t route via the public internet. How is a hacker going to compromise this system?

    Reading your pulse. So you’re going to hire someone to hold 3 fingers on your wrist while you sleep are you? I’d prefer a camera personally.