On Saturday I stopped by the Apple store and picked up an UP device by Jawbone. The UP is a wristband and iPhone app that will track your fitness and sleep activity and help encourage you to live a more active life style. It’s very similar to the Fitbit device, and lets you track:

• steps
• distance
• calories burned
• active/inactive times
• workouts
• sleep (awake/light/deep sleep)

So far I’m really impressed with both the wristband and iPhone app.  However, I was a little disappointed the wristband didn’t support ANT or Bluetooth for data synchronization.  The data sync is unique and “low impact” (battery-wise) in that it plugs into the iPhone via the headphone jack and uploads data directly to the app. After some digging I was surprised to see no way to send or save the data.  I was curious if the data was just saved to the app, or also synchronized with another server.

As with most of my Quantified Self gadets (Fitbit, Zeo, Withings, Neurosky, etc.) my first reaction was to “free my data”.  My understanding is that Jawbone has exposed an API to partners, and will soon be releasing a more public API and ability to download data shortly.  Similar to my initial journey with the Fitbit, I was inspired to find an “unofficial” way to get access to the data (e.g. my Fitbit Hacks).  I was able to get access and save my personal data for about a year before an official method became available.  Luckily Fitbit and other companies like Zeo (especially Zeo) have taken an open approach with their APIs and have embraced the developer community.  My hope is that UP will do the same.

In the spirit of filling in the data gaps until that day comes and an official announcement is made I decided to do a bit of exploration yesterday morning.  Using my favorite HTTP/SSL sniffer (Charles) and some tinkering with the app I was able to decipher how to get access to all of the UP data.  I haven’t written a data scraper script yet, but I was able to verify a number of API requests from the command-line using LWP POST and GET.

I wrote up some detailed notes in my wiki yesterday and have posted a snapshot here:

Jawbone UP API – http://eric-blue.com/projects/up-api/

My hope is this should give others enough information to download their own data for the purpose of backing up and analysis with other software.  Since this isn’t officially supported and appears to be used only by their app I would use sparingly (no excessive requests) and don’t do anything that would violate a TOS/AUP (use at your own risk).  Otherwise, enjoy!

My other Quantified Self-related projects:

TRAQS.me (Tools for Reporting & Analysis of the Quantified Self)

Fitbit Hacks (the original “unofficial” API)

Zeo (Hacking your way to a better night’s sleep)

NeuroSky Brainwave Visualizer

Mindstream – Neurosky EEG Data streamer

Devices like FitBit make syncing and importing data relatively fast.  As soon as the device syncs with the ANT-based station, data is uploaded to the FitBit site and this can be easily extracted.  However, for Garmin devices like the FR60 heart-rate monitor, extracting data from the Garmin Training Center product is a manual process.  You have to manually launch the app, go to Export, and save your data as a .TCX file.

You can pick and choose each activity to export (tedious), or you can save the entire data (a larger file ~ 15MB of data now) which takes some time to save and process.  This is clearly less than ideal.  In an effort to have my data INSTANTLY imported and available, I spent some time researching ways to accomplish this.  The hacker in me was thrilled when I discovered a linux-based open source project called GarminTools.  From the project page:

“This software provides Linux users with the ability to communicate with the Garmin Forerunner 305 via the USB interface. While this is the only Garmin unit that I own, I did implement all of the documented Garmin protocols as of Rev C (May 19, 2006) over the USB physical link. This means that if you have a Garmin with a USB connection to a PC, you ought to be able to use this software to communicate with it.

If you’re looking for a complete solution to all of your Linux Garmin GPS needs, this is not it. I own a Garmin Forerunner 305 and wrote this code specifically so I could download and save data from that particular GPS unit to my Linux machine. I tried gpsbabel, but found that it did not have command line options specific to the Forerunner run and lap data – all I could do was get the tracklog. That’s how all of this got started. I also wanted a few other things, like the ability to convert a track log into a Google maps encoded polyline, and (eventually) the ability to generate PNG images of heart rate and elevation data.”

I’m going to eventually dig deeper into this project, but since my time is more constrained this weekend I opted for a more pragmatic (ok cheap) hack.  I discovered that the Garmin Training Center app stores individual TCX files each time you sync under a folder (on Windows 7) like:

C:\Users\username\AppData\Roaming\GARMIN\Devices\1110044\History

I figured a simple approach would be to sync this folder with my server, find any files newer than the last imported Activity/Heartrate data, and process.  A simple windows share could have sufficed, but I figured it might be easier just to sync with Dropbox and make the data available from any of my computers.  So, here’s the quick n’ dirty recipe:

1) If you don’t already have an account, signup for Dropbox and install clients on your computer running Garmin Training Center, and where you ultimately want to send your .tcx files.

2) After your Dropbox folder is setup and you’ve done a simple test to verify sync works, you’ll want to share this folder.  Now, I’m sure there’s an easy way to share folders OUTSIDE of the Dropbox folder, however 1 minute of Googling with no luck made me look for another option.  On Windows-based machines you can actually do the equivalent of a Unix symlink using the Junction program from sysinternals.  Here’s how to set it up:

• Download and install Junction from the URL (you can copy the junction.exe file to c:\windows\system32)
• Go to the command prompt (cmd), cd to your Dropbox directory, and run:
• c:\windows\system32\junction GarminHistory C:\Users\username\AppData\Roaming\GARMIN\Devices\1110044\History
• Do a directory listing and verify the new GarminHistory folder exists under Dropbox and syncs properly

3) Go to your destination server where Dropbox is installed and presto, you have all of your workout/activity/heartrate data available in .TCX format!

Since last August when the project was originally envisioned, I’ve made a lot of progress on not only building this system for my own use, but also expanding on the original idea of just ‘tracking fitness’.  Fitness is one significant component of improving oneself and general well-being.  But, when most people think of fitness, it’s associated with simply going to the gym, running on the treadmill, or pumping iron indoors, and tracking these activities in some journal.  The scope of my project has since grown to include tracking for sleep quality, detailed location tracking and geocoding, and heart rate monitoring.  I’d like to eventually expand the scope further to include nutrition tracking, health metrics (blood pressure, nutrient levels/blood tests, cholesterol, etc.), and even more subject things like mood and factors that affect emotional well-being.

With this in mind, my personal project has evolved into a more general purpose tool for tracking and analysing data related to the Quantified Self.  Hence, my new project name TRAQS.me (Tools for Reporting & Analysis of the Quantified Self) or simply put, it’s a tool that (Tracks “Me”).  I’ve put together a slideshow/presentation that gives a decent overview of the entire project:

What’s Been Done So Far?

If you look at the original diagram I created, there where a few different components to this tracking platform (local storage, data collection scripts/api integration, and the presentation layer).

• Local Storage – Personal copy of data stored in MySQL.  I created the local storage piece almost immediately after writing the article in Aug.  I chose to use MySQL as a database, and created a schema that was suited towards collecting my initial data set of interest: data from my FitBit and WiThings scale.  The schema accounts for storing both daily and intraday activity for calories burned/consumed, activity score (+very active, moderately active, and lightly active), total distance, steps taken, sleep, and weight.  I’ve now created about a dozen or so SQL queries that will generate useful reports on this historical data.

• Data collection scripts/API Integration. I immediately leveraged the Perl FitBit API I originally wrote last April to sync data from the FitBit site to my local database.  I created a nightly cron job that will request the daily + intraday stats and save to the db using Perl::DBI.  I also took a similar approach with the WiThings scale.  Although I haven’t created a Perl wrapper yet for Withings, they do publish and easy to use REST API for getting access to your weight.  This script also runs on a nighly basis and saves weight information to the local database.

• Presentation Layer.  This has been the most complicated piece so far.  I’ve created a Java-based webapp using Spring 3.0/Jetty to build the analytics UI.  For now the UI is built with 2 major views: a dashboard, and reporting interface.  The dashboard displays both historical and daily information in a visually-pleasing format using Google Charts.  You can see a daily snapshot (+ weekly, quarterly, and yearly views) and see stats on your activity, distance, steps taken, weight, sleep and intraday stats (activity and sleep level on an hourly basis).  I’ve also built a light home-grown reporting interface that integrates with the Google Visualization API and easily builds dynamic tables and charts based on the provided SQL queries.

Screenshots

• Dashboard View

• Historical Stats – Multiple Activity Logs

• Historical Stats – Distance Bar Chart

• Daily Stats

• Intraday Stats

• Reporting

Immediate Next Steps?

The current code is fairly stable and mostly ready for a demo.  Before I make it live I’m planning on adding a few major features:

1. Geo-Location support – I’ve been tracking GPS coordinates since I purchased my QStarz device in September.  I’m thinking of adding a couple Map and GPS tracklog views to the Dashboard.  And, this could lead to some interesting location-based reporting as well.  I have a ton of saved .gpx files, so it’s a matter of writing an integration script to parse and store in the db.  And, some lightweight presentation layer to dynamically generate .KML and overlay on top of Google Maps.
2. Simple mobile interface.  I’d like to be able to access the interface using my iPhone and having some light interface to get quick stats and graphs would be nice.
3. Come up with a catchy project name.  As I stated earlier, although the original scope of this was for fitness tracking, this type of platform can be used for tracking any type of personal statistics.

Future Roadmap

1. Tracking detailed sleep info from MyZeo.  Since I wrote the original vision I had the chance to do a review for the MyZeo sleep device.  I’ve also created a Perl API so it’s simply a matter of writing the integration script to start recording in the MySQL DB.
2. Tracking workout info from my iPhone app – iFitness.  I also have about 10 years worth of workout information either stored from previous products (like CrossTrainer) or written down.
3. Tracking heartrate info from my Garmin heartrate monitor watch.
4. Integrate with the new official Fitbit API.  Since I wrote my initial (unofficial) Perl API, fitbit has embraced the developer community and released a support API (yes!)

What’s a better way to kick off the New Year than by giving tips on how to improve your overall quality of sleep? I decided to write this post as a mini-review of the Zeo Personal Sleep Coach and share some open source tools and APIs to help with saving your Zeo data for your own personal analysis.

Overview

In retrospect, 2010 was a pivotal year for me since I first learned about some amazing new technology gadgets that help you keep track of your overall health and well-being. Last February I discovered the WiThings scale that will automatically track your weight and BMI, and will upload your data online so you can set goals and visually see how you are progressing. Then, in April I received my FitBit and have been consistently using it to track my overall fitness activity – calories burned, miles walked, activity level, etc. The thing that I love about these gadgets is that they “just work”: simply stepping on scale, or wearing a little gadget on your belt will help produce interesting sets of data and visual graphs that can help guide you to take action to improve your health. A few months ago I discovered another amazing device called the Zeo Personal Sleep Coach that will actually track how well you’re sleeping, provide detailed data and visual feedback, and give coaching tips to improve your overall quality of sleep.

What is Zeo?

In a nutshell, the Zeo product is designed to analyze your sleep patterns and provide you with a lightweight wireless headband, a bedside display, a set of online analytical tools, and a personalized email-based coaching program.  The main thing that initially blew me away with Zeo was the fact that the wireless headband will actually record your brainwave activity!  Not only can the device tell you what time you go to bed and what time you wake up, but it will provide detailed info on exactly when you’re lightly sleeping, in REM sleep, and in deep sleep.  With all this info at its disposal, the Zeo is able to provide you some valuable information and tools to analyze trends in your sleeping patterns.  Here’s a sample of some of the sleep data you can review once it’s been uploaded to the myzeo.com site:

• Zeo Sleep Score

When you wake up and place the headband back on the bedside display, the Zeo will provide you with an overall sleep score (ZQ) that is based on your sleep quality and quantity.  Your base score is your total sleeping time, combined with restorative sleep (Deep & REM) and disrupted sleep (wakeups, total wake time).  Think of the score as your overall grade for how well you slept.  I was surprised to learn that the average score for my age group is a ZQ score of 77.  Since I started using the Zeo in November, my sleep score has been averaging a ZQ of 98.

• Sleep Graph

This is where the Zeo data can get interesting.  This particular graph will not only show what time you went to bed and woke up, but also the total time it took to fall to sleep and a detailed breakdown of what sleep phase you were in throughout the night!  Reviewing this graph can provide some telling feedback on how different factors may impact your night’s sleep: going to bed too early/late, stress (a racing mind), too much caffeine or alcohol, working out too late in the evening, etc.

• Sleep History

The sleep history graph will show key information (ZQ score, total sleep time, deep sleep/rem sleep, etc.) for different time intervals (daily, weekly, monthly).  Again, being able to use this data to see trends in your sleeping patterns can provide some useful feedback on how to be more conscious of what’s impacting your quality of sleep.

For a good review and video of the Zeo, I’d recommend reading the WallStreet Journal article from last year (note the price for the Zeo is now $199.95 USD and available at retailers like Brookstone & Sharper image). And, for another recommended review with some detailed info on how the Zeo actually works check out Ryan Hurd’s review and some peer-reviewed literature on the science behind Zeo.

Open Source and APIs

Now, this is where the post gets a little technical (and exciting)!  The thing that I *really* like about the Zeo is their willingness to give you as much access to your sleep data as possible and embracing open source as a compliment to their commercial offering.  How are they doing this?  Zeo issued a press release back in June announcing their new open API to allow 3rd party companies and individual developers complete access to their sleep data.  The published API offers developers the ability to register and signup for an API Key, and provides a comprehensive REST API (using both JSON and XML) so 3rd party libraries can be developed.

Zeo currently offers some sample code using the API in a variety of languages (Perl, PHP, Java, etc.) and a fully working Ruby libarary developed by DailyBurn.  In order to help ease my own effort to download data and help other developers, I decided to create my own OO Perl library similar to the one I initially created for Fitbit back in May.  Unlike the FitBit libary which is somewhat unofficial (Fitbit does not offer ‘published’ API for developers), the Zeo Perl library has been fully tested against the published API, and any future changes by Zeo can be easily incorporated in the the library.

The Perl API (WebService::Zeo) is now hosted on GitHub at http://github.com/ericblue/Perl-Zeo-API. In the near future this library will also be made available on CPAN. This module provides an OO (Object-Oriented) library that currently supports all major services in the REST catalog (methods for getting sleep averages, detailed sleep data based on exact dates and ranges, and pagination to step through the sleep records).

More Open Source “Hacking”

In my mind, the officially supported API is major news for techy geeks and data junkies (me). But, it gets even cooler…. along with the Web API, Zeo also offers:

• Zeo Data Decoder Library – A version of software and a library that enables a user to decode Zeo sleep data stored on a memory card without uploading data to mySleep.myZeo.com.
• Zeo Raw Data Library – A version of software and a library that enables a user to decode Zeo sleep data and raw signal data in real time using the auxiliary serial port located on the back of Zeo.

Their developer site lists some cool sample projects:

• Integrate Zeo sleep data into a website that tracks your weight, nutrition, and exercise.
• Create a desktop application to view and process Zeo sleep data.
• Build a lucid dreaming system to notify you when you are in REM sleep and alert you using a sound.
• Use Zeo while you are awake to measure meditation.

Overall, I’m very pleased with the Zeo so far and would highly recommend to anybody interested in improving their sleep quality.  And, for the “hackers” out there, the open source offering provides some great tools to access all of your sleep data and analyze in unique and interesting ways.  Happy 2011, and I wish all of you restful sleep and a healthy ZQ score!

Late last year I wrote a personally pivotal blog post on Total Recall, Personal Informatics and Life Logging.  This was an eye-opening post for me since I discovered the work of Gordon Bell, learned more about his MyLifeBits project which parallels my own Personal Memex project (albeit MyLifeBits was started 10 years ago and has quite a head start) and picked up his book Total Recall.  The book has reaffirmed my belief that a need for an intuitive and consumer-friendly personal memex is on the horizon, and made me aware of a number of devices on the market today that that help assist with e-memory and personal informatics.   From the article on what personal informatics is all about:

“(Personal Informatics is)… characterized as the monitoring and displaying of information about our daily activities through intelligent devices, services and systems. This information allows us to see trends and opportunities for change that we would otherwise miss.  With the rise in network and RFID technology we are pointing to a time where personal informatics can play an important role in our lives. If people can access this information about their daily routines, and interact with their own personal data currently invisible to them: would they make more informed decisions?”

And, from the PersonalInformatics website:

“Personal informatics is a class of tools that help people collect personally relevant information for the purpose of self-reflection and self-monitoring. These tools help people gain self-knowledge about one’s behaviors, habits, and thoughts. It goes by other names such as living by numbers, personal analytics, quantified self, and self-tracking.”

Immediately after reading these articles, and further researching in the book, I decided to start experimenting with some of these devices.  Although there are a lot of different type of lifelogging and personal informatics devices, the area that caught my interest for a practical application is for fitness.  Having devices that can automatically track your weight, fitness activity, and workouts and keep you informed on what decisions you should be making is powerful.

I’ve already evaluated the FitBit device for tracking fitness activity (calories burned, miles walked, sleep patterns) and have successfully created an API for pulling this information directly from fitbit.com (See my Fitbit Hacks projects).  I’ve also been using the WiThings Wi-Fi Scale for the last 6 months.  This scale is amazing in that as soon as you step on the scale it will automatically connect over Wi-Fi and upload your weight, BMI, and lean mass info to the WiThings site which can be viewed on your PC or iPhone.  For workouts I’ve been using the iFitness application for both the iPhone and iPad and love the simplicity of tracking workouts.  And, I just recently came across the Zeo Personal Sleep Coach which provides very detailed information on the quality of your sleep (I haven’t tried it yet, but it looks amazing and I’ll be getting a device shortly and will evaluate).

All of these devices (combo of hardware and software) have a couple things in common:

* They make tracking personal information VERY easy.  In most cases they just work automatically (e.g. step on scale, clip device onto your belt, etc.).

* Data is ready made available for viewing: you can see trends via graphs and make informed decisions

But, there are also some major downsides:

* Getting data from these 3rd party websites is not always easy

* There is no cross integration (e.g. Weight gathered from WiThings can automatically be uploaded to Fitbit or my workout logs, etc.)

I’ve been thinking about ways to integrate all of this data from various products into a single interface.  My ultimate goal is to make this a sub-project of my Personal Memex project.  All areas of your fitness and health could theoretically be tracked from a single place.  I created a quick diagram to illustrate (Large diagram also available on Gliffy):

So, this looks fancy, but what are the practical applications?

1) Keeping your weight under control.  Being able to see your trends, coupled with current activity (coming from FitBit and workout sheets) will allow you to gauge how active you need to be.

2) Staying motivated to workout.  Again, being able to see trends on how many times a week you’re going to the gym or going outside to be active (walking, hiking, running) and getting real-time notifications is a way to help you stay on track.

3) Observing your sleeping patterns to see how it relates to your overall stress and productivity.

This may seem like a high degree of information logging and complexity for most people.  But, I think the real benefit is this: Low-maintenance and High-reward = Great results.  Most of these devices are already tracking this information, and with minimal development, systems could be built to collect, aggregate, and present this information in ways that can help you shift your behaviors and actions.  These are all preliminary ideas right now, but my hope is to start prototyping this over the coming months, integrate into my existing personal memex system, and then release this proof of concept as an online demo.

The Fitbit team is working hard on manufacturing and distribution and have future plans for releasing an official API. Although the website does offer most of the features you’ll need, you can not download your data. The goals is to have an API that will allow you to do this and also integrate some other applications. I spent some time coding last weekend and have create an OO Perl API to access all Fitbit information and even download historical data in CSV format. I’ve setup a ‘Fitbit Hack’ project. Check out:

http://eric-blue.com/projects/fitbit/