Being able to access your personal information/knowledge from multiple places is the ultimate realization of total information ubiquity.  Being able to access all of your personal bookmarks, notes, contact information, journal entries, and research data from any computer is obviously useful.  Being able to access all of your personal knowledge from a handheld device like an iPhone is absolutely exciting!  Without sounding totally nostalgic, this type of portability is in a large part a modern-day realization of what Vannevar Bush had envisioned in his article on the Memex (“As We May Think“).

“Consider a future device for individual use, which is a sort of mechanized private file and library. It needs a name, and, to coin one at random, “memex” will do. A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.”

Currently, my personal wiki and other research data (bookmarks, pdfs, mindmaps, etc) are stored on my private server (accessible only behind my firewall).  Unless I enable SSH access, my wiki content is not generally available from the Internet and I have no way to easily access remotely.  I’ve been thinking for a while on the best approach for making this data completely portable.  After some experimentation, I’ve found an easy method for making my personal wiki completely accessible in offline mode right on my iPhone.  At a high-level all you really need to do are 2 things:

1) Find software that can take a snapshot of your wiki content and make it available for offline viewing

2) Find software that lets you save a copy of your snapshot wiki, store on the iPhone, and view in a web browser (actually both on the phone itself and another PC)

Creating a backup of your wiki

There are a lot of applications out there that act as ’spiders’ that crawl your website and save local copies of your pages so you can view in offline mode (no need for an Internet connection).  After trying a handful, one of the better applications I tested was HTTrack (available for Windows and Linux). I should note that I really did try to make this work with Scrapbook.  To date I’ve used Scrapbook to capture copies of pages with no problems.  However, it turns out that backing up a wiki pushes it to its limit… Scrapbook only does one serial http connection at a time, doesn’t have a configurable delay between requests (default is 1 sec and this takes too long), filtering options are not extensive enough, and the process of dynamically updating the HTML to support relatives links took way too long.  In the end, HTTrack ended up being the best solution for a complete wiki backup.

HTTrack is a highly configurable crawler that allows you to create a complete snapshot of your wiki (Mediawiki in my case).  Crawling a wiki turns out to be a little more complicated that your typical website.  Because wiki’s offer a number of functions (editing of pages, viewing history, printing and exporting in other formats) there are certain links that should not be included in the backup.  After some trial and error, I discovered that since I used Semantic Mediawiki I needed to be even more careful with the links I wanted to include (many of the Special and Property pages took FOREVER to index).

I tried the windows version of HTTrack (even under Wine on Linux) and the web client version as well.   However, was not completely impressed with how it worked.  What I wanted was a command-line script to run the backup.  Luckily, I found a couple websites that have used HTTrack for this purpose and decided to use for my own needs.  Here is a copy of the script i used to create the offline snapshot of my wiki:

Code block

#! /bin/sh
# Inspired by blogpost from http://www-public.it-sudparis.eu/~berger_o/weblog/2008/05/30/offline-backup-mediawiki-with-httrack/
# -w mirror web sites (--mirror)
# -O backup directory
# -%P extended parsing, attempt to parse all links, even in unknown tags or Javascript (%P0 don't use) (--extended-parsing[=N])
# -N0 Saves files like in site Site-structure (default)
# -s0 follow robots.txt and meta robots tags (0=never,1=sometimes,* 2=always) (--robots[=N])
# -p7 Expert options, priority mode: 7 > get html files before, then treat other files
# -S Expert option, stay on the same directory
# -a Expert option, stay on the same address
# -K0 keep original links (e.g. http://www.adr/link) (K0 *relative link, K absolute links, K3 absolute URI links) (--keep-links[=N]
# -A25000 maximum transfer rate in bytes/seconds (1000=1kb/s max) (--max-rate[=N])
# -F user-agent field (-F "user-agent name") (--user-agent )
# -%s update hacks: various hacks to limit re-transfers when updating (identical size, bogus response..) (--updatehack)
# -x Build option, replace external html links by error pages
# -%x Build option, do not include any password for external password protected websites (%x0 include) (--no-passwords)
site=wiki:8080/memex
topurl=http://$site
backupdir=~/websites/memex
httrack -c4 -w $topurl/Special:Allpages \
-O "$backupdir" -%P -N0 -s0 -p7 -S -a -K0 \
-F "Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)" \
-%s -x -%x  \
"+*$site/index.php?*" \
"+*$site/mindmap*" \
"-*Special*" \
"-*Property*" \
"-$site/index.php?title=Property:*" \
"-$site/index.php?title=Special:*" \
"-*$site/Discussion:*" \
"-*$site/Help*" \
"-*/docs/*" \
"-*/wikifiles/*" \
"-*month=*&year=*" \
"-*action=edit" \
"-*action=formedit" \
"-*action=history" \
"-*printable=yes" \
"-*oldid=*" \
"+*$site/images/*" \
"+*.css" \
"+*.js"

The best feature of HTTrack is that it will download all content, including Javascript and Flash, and update all links and make them relative.  This way the entire website can be viewed offline and made portable.  Overall, my entire wiki backup was ~ 30MB of markup and content (of course, excluding all audio and video).  And, in the future I need to come up with a solution for exporting my mindmaps.  Since iPhone does not yet support flash I’ll need some other way to allow for embedded viewing of my mindmap content.  Anyways, at this point all of my content is now ready for copying to my iPhone.

Storing my Wiki on my iPhone

OK, this is the really nifty part.  The one thing I really missed about my old 60GB IPod Video was the ability to mount it over USB and use it just like an external hard drive.  I used to haul around TONs of my data and could easily share between Windows, Linux and Mac.  Unfortunately when the iPhones and iPod Touch’s came out, you could no longer mount the iPhone and copy files (without hackery of course).  Luckily there are a number apps that let you use your iPhone as a storage device.  One of the BEST applications out there is an app called AirSharing.

With Air Sharing, you can:

• Mount your iPhone or iPod touch as a wireless drive on a Mac, windows, or Linux computer, over Wi-Fi, or connect from your computer’s web browser.
• Drag-drop files between your iPhone or iPod touch and your computers.
• View documents in many common formats.

What’s really useful is that you can mount your iPhone using WebDAV and transfer files just like a regular drive.  The incredibly cool bonus is that you can also access your content from another computer.  If you’re connected to the same Wi-Fi network, you can use any PC to browse (e.g. http://iphone-local:8080/wiki/) and access your content just like it was on the original server.  For an added layer of security, while you’re on the go you can setup an AdHoc wireless network and connect privately between your computer and the iPhone to access your personal knowledge base.

Of course, accessing the content on your iPhone from another PC is an added bonus.  The real power in this solution is the ability to browse your wiki on the iPhone without needing any Internet access (3G Or WiFi).  Simply open up your Airsharing app and browse directly to your wiki folder and click on index.html.  Wala!, your browsing your personal wiki just like usual.

I exported the majority of the text content from my wiki (preserving the original formating, with Javascript support).  In fact, I even shared my digital scrapbook that I blogged about last week.  but you can also choose to export your entire document collection and multimedia files (video, MP3s, etc).  This is incredibly useful for taking your knowledge on the go and having all of your data RIGHT at your finger tips.  Here are some screenshots of my personal knowledge manager wiki right on my iPhone:

All Articles

Workout Journal

Learning

Documents

Background

For my own personal knowledge management setup, I’m very interested in tracking a number of different ‘things’:

* Documents – PDFs, word documents, mindmaps, etc.

* Notes – Journal entries, book summaries, personal notes (think wiki text)

* Links – Bookmarks (personal or social sites like del.icio.us)

* Multimedia – Audio / Video

* Snippets – Captured web pages (full or partiallly snipped content)

When I first mentioned my ‘Digital Scrapbook’, I wasn’t dropping any hints about me having any crafty hobbies, I generally refer to my system for storing Snippets as my Scrapbook.  This name is no doubt in large part due to the fact that I’ve been using the popular Firefox plugin ScrapBook to manage my digital snippets for a few years now.

ScrapBook is a fantastic solution for storing local copies of web pages for research (with highlighting, editing, and annotation), saving snips of important sections of sites, recording purchase confirmations or receipts, and saving your travel itineraries.  One major thing it has been lacking though is the ability to synchronize or share the Scrapbook with other computers. I use multiple computers (a couple laptops: Mac & Window and a central desktop: Linux) so my goal is to have consistent and up to date data between all systems.  And, up until now, I’ve had no way to integrate this save data into my wiki-based knowledge management system.

I started investigating a solution for this a number of months ago and stumbled across a related (and powerful) research tool called Zotero.  I haven’t had a chance to use Zotero in depth, but one new feature in the beta version that stuck out to me was the ability to synchronize your data with a remote server.  On the surface this feature looks good (and probably is for most people – data sync to Zotero server and webdav support for documents), but I was looking for a solution where I have more control over where the data is hosted.  Although I’m usually not concerned with hosting my data with most providers, I often save private financial information in my Scrapbook (credit reports, financial statements, account numbers, etc.) so I’d like to have control over where the data is saved and how it’s encrypted.  Further research eventually sparked a few ideas for a solution.

Synchronizing and Sharing ScrapBook Data

I decided to find a way to explore a setup using some file sharing/sync services after reading an article on syncing Scrapbook using Dropbox.  I had never used Dropbox before and after giving it a brief testdrive it looked very promising.  Hey, you get a 2GB account for free so that’s definitely an added bonus!  Although Dropbox has some killer features (a big one being an iPhone app to access your files), I opted to experiment with another sync service.  I’ve been using JungleDisk for a couple years as my Amazon-S3 backed offsite backup solution, and was curious if this could be used.  After downloading the latest version (3.0.2 for Linux)  I discovered that it now supports file/directory synchronization between computers.  After about 10-15 minutes of setup and file syncing I had a working solution between my laptop and desktop computers.  Here’s what you’ll need to do:

Step 1: Download and install the latest version of the Scrapbook plugin for Firefox on your 1st computer.  For a good quick intro/tutorial to Scrapbook, check out this video from Lifehacker.

Step 2: Setup an alternate Scrapbook location that resides outside of your Firefox profile directory (Prefrences -> Organize -> Save data to)

Step 3: Setup your preferred sync solution and use the directory provided in Step 2.  I preferred JungleDisk for my setup, but there are other services like Dropbox, Box.net, SugarSync, etc.  Check out the Activty Owner wiki for a detailed list of sync services.  And, although I haven’t personally tried yet, I’m sure there are some other non-hosted open source sync solutions like Unison (cross-platform) that could be used.

Step 4: For your 2nd (or subsequent computers) repeat steps 1 through 3.

Wiki Integration (Extra Credit)

OK, for me this was the icing on the cake.  Since my Scrapbook data is now on the same computer as my wiki I thought it would be nifty to somehow integrate directly into some of my wiki pages.  I found out that Scrapbook supports the ability to export your Scrapbook hierarchy as a tree in HTML (from Scrapbook Sidebar: Tools -> Output Tree as HTML).  Although this isn’t completely automatic (yet) this gave me the the content I needed to add to my wiki.  Now, since wikis by there very nature dont’ typically allow you to embed other HTML pages I needed to find a way to make this work.

Step 1: Setup a directory on your webserver to serve content from your Scrapbook directory (setup in Step 2 above) (e.g. http://yourwebsite/scrapbook).  This can either be on the same server as your wiki or another, it doesn’t really matter.

Step 2: Verify the output of the directory tree looks good.  If you enabled frames, the URL should be something like http://yourwebsite/scrapbook/tree/frame.html.

Step 3: For MediaWiki users there are various ways to directly embed pages in your wiki content.  I found that the AnySite extension did the trick for me.  Enable the extension, pick a wiki page where you want to display your ScrapBook data and you are set!  For example, here is my content:

* Link to [http://wiki:8080/wikifiles/scrapbook/tree/frame.html ScrapBook Tree]
<anyweb mywidth=”1024″ myheight=”768″>http://wiki:8080/wikifiles/scrapbook/tree/frame.html</anyweb>
[[Category:Documents]]

I also discovered a somewhat related research project at Stanford called Vispedia.

“Imagine taking a unique journey through the depths of your mind and stepping into a world of limitless possibilities where every one of your thoughts, habits, behaviors, beliefs, decisions and actions are revealed to you in vivid detail. Within this scenario “causes“ are identified and “effects“ are measured, effectively unlocking the hidden patterns that are continuously re-creating your life experience on a daily basis.”

What is the MasterMind Matrix?

“In the simplest terms, the MasterMind Matrix Chart presents you with a visual Concept Map of Your Personality. It progressively unlocks and expands your understanding of your current patterns of behavior, while helping you develop new habits that can guide your life in the direction of your choosing.”

You can download the free version of the map, or read about it in depth and buy the premium version ($50USD).

Most traditional personal knowledge management (PKM) or personal information management (PIM) applications offer the same basic set of features:

* Storage of notes and documents
* Search functionality and keyword/tagging capability
* Outline view in a traditional hierarchy, or user-defined views
* Task management, calendar, and contact management (mainly PIM, not KM)

These are essential features, however don’t offer too much to the more visually-inclined knowledge junkies. For visual learners and information visualization fans, having a graphical representation of knowledge and seeing how things relate is a must have feature.  Luckily, in the past few years there has been a rise in the number of knowledge management applications that offer this capability.  The following is a list of interesting /unique / effective tools for knowledge management and information visualization (not listed in any particular order):

15. DeepaMehta

Link: http://www.deepamehta.de/
Platforms: Win, Mac, Linux
Cost: Free (Open Source)

DeepaMehta is a “networked semantic desktop” that replaces the traditional computer desktop. DeepaMehta rids the user from dealing with applications, files and directories. Instead, the DeepaMehta user arranges information of any kind and origin into supportive topic maps. Topics may be e.g. projects, emails, webpages, notes, articles, contacts, or meetings. Users define their own topic types. Old-fashioned applications, windows and files are no longer in the sight of the user, but the meaningful relationships between real world topics.

14. Tinderbox

Link: http://www.eastgate.com/Tinderbox/
Platforms: Mac
Cost: $179

Tinderbox stores and organizes your notes, plans, and ideas. It can help you analyze and understand them. And Tinderbox helps you share ideas through Web journals and web logs. Tinderbox maps your notes as you make them. Build relationships by arranging notes, organizing them with shape and color, linking them. Tinderbox lets you record ideas quickly and keep them where you’ll find them again when you need them.

13. Vue

Link: http://vue.tufts.edu/
Platforms: Win, Mac, Linux
Cost: Free (Open Source)

At its core, the Visual Understanding Environment (VUE) is a concept and content mapping application, developed to support teaching, learning and research and for anyone who needs to organize, contextualize, and access digital information. Using a simple set of tools and a basic visual grammar consisting of nodes and links, faculty and students can map relationships between concepts, ideas and digital content.

12. eyePlorer

Link: http://www.eyeplorer.com/eyePlorer/
Platforms: Web
Cost: Free

EyePlorer allows you to explore and process knowledge. Search engines help you find links and documents – they require you to follow these links and open the respective document in order to access information. eyePlorer.com, powered by vionto, provides immediate access to facts. It visualizes facts as well as relationships between facts. Furthermore, eyePlorer.com allows you to collect, process and publish interesting bits of information. eyePlorer.com is a visual knowledge workbench.

11. BeeDocs Timeline

Link: http://www.beedocs.com
Platforms: Mac
Cost: $65

Bee Docs Timeline is software for Mac OS X that makes it easy for you to present historical events in a way that reveals connections and clarifies relationships.

Make timeline charts of world history, family trees, fictional events or business deadlines. Timelines can help you understand and present history with new perspective!

See: http://eric-blue.com/2009/04/14/how-to-create-stunning-3d-timelines/

Post continued, click below to navigate to the next page….

I have to admit, when I first came across this software last week I was instantly impressed.  I’ve had a fascination with software that allow you to visualize and comprehend information in different and exciting ways.  Bee Docs Timeline, an application for Mac OS X (sorry Windows and Linux fans), is a great example of how to create and present smooth, professional looking timelines in a relatively easy fashion.

Here’s a brief summary of their product:

“Bee Docs Timeline is software for Mac OS X that makes it easy for you to present historical events in a way that reveals connections and clarifies relationships.

Make timeline charts of world history, family trees, fictional events or business deadlines. Timelines can help you understand and present history with new perspective!”

This application has a number of features, the most compelling being the ability to view the timeline in 3D, and even save it as a movie (Quicktime).  Not only can you create and import traditional timeline data, but you can also import from iPhoto, iTunes, and RSS feeds.  For illustration purposes, I created a 3D timeline video of my blog posts for the last few months (albeit less frequent posts than normal thanks to Twitter).

“One of the great things about an iPhone or Android phone is being able to play Pacman while stuck in line at the post office. Sometimes though, we yearn for something more than just playing games or watching videos.

What if you could also access literature’s greatest works, such as Emma and The Jungle Book, right from your phone? Or, some of the more obscure gems such as Mark Twain’s hilarious travelogue, Roughing It? Today we are excited to announce the launch of a mobile version of Google Book Search, opening up over 1.5 million mobile public domain books in the US (and over half a million outside the US) for you to browse while buying your postage.”

Continue reading at http://booksearch.blogspot.com/2009/02/15-million-books-in-your-pocket.html

Abstract. The use of semantic representations in document-oriented
environments – as formal annotations or embedded instances of a formal
knowledge base – is seen as an enabling technology for intelligent services
which may help knowledge workers in tasks like finding, structuring, or
assessing information. A very high level of formalization may even more
directly support problem solving, e.g., by the application of inferencing
services. A coupling of textual and formal representations in document-
centered knowledge work raises, amongst others, two questions: i) How
can the acquisition of formal knowledge in such an environment be facili-
tated? ii) How can the potential complexity of formal annotations during
a document’s life and use cycle be adequately handled?
We present the Mymory workbench as an approach to tackle these chal-
lenges. Mymory is based on a semantic wiki system and supports manual
as well as automated annotations of wiki documents. These annotations
can be framed by automatically obtained models of the user’s work con-
text, thereby a situation-oriented structuring of the annotations is being
achieved which can be exploited in semantic search and adapted docu-
ment presentation. In the paper, the general approach and architecture
of the system is being described. Example use cases show how document-
centered knowledge work can look like in such an environment and give
an idea which services can be possible with contextualized fine-grained
annotations.
2008_EKAW_Mymory

I’ve created an initial project page outlining the vision, current applications, and future areas of research.  This page should be considered a living document, and I plan to add more detail as I make progress.  I would love to hear feedback.

The Vision

Over the last few years, I’ve been interested in the fields of Personal Knowledge Management (or PKM) and Personal Learning Environments (or PLE).  I’ve been a knowledge worker as long as I can remember, and have subsequently searched long and hard for numerous software applications and systems to help me better process, organize, and retrieve information.  I’ve tried many different free and commercial solutions (outliners, PIMs, personal knowledge bases, mind mapping software, notebooks/pad, etc.), but none of then were 100% complete in my mind.  Then, In 2006 I came across Vannevar Bush, and learned about his amazing vision for the Memex (or “memory extender”).

Read more at: http://eric-blue.com/my-projects/personal-memex/

It’s about time I spread the news about this truly great site!  Brian Johnson, entrepreneur and philosopher at heart, has created a new site/service called PhilosopherNotes.  What is PhilosophersNotes?

Think: Mini-CliffsNotes for Self-Development Books!

“Brian’s always been passionate about understanding what makes great people great and applying the truths they embodied to make his little dent in the universe.

After selling his last business (Zaadz) and traveling for a bit, Brian decided to give himself a Ph.D. in Optimal Living with a Specialization in Greatness and Bliss.

He figured he’d get a Master’s when he’d distilled 100 of his favorite books into concise, fun, inspiring summaries. Then, on a flight from Tokyo to LA, he had one of those “Aha!/hallelujah!!” moments when he opened the SkyMall to an ad for a company that summarized business books for “busy executives.” He decided someone should do summaries of self-development books for the “busy self-actualizer” and that that someone should be him. ☺”

I’ve gone through quite a few of the summaries so far(6 pg PDFs + 20 min MP3s), and have to say I’m super impressed!  Now, here comes the great news….. you can signup and get 25 titles ….. for FREE!  Yes, free.

Click here to get signed up.