Who doesn’t wish to keep a record of a beautiful sunset that particularly impressed us in childhood, our first kiss or, for that matter, an important conversation with the boss that took place a few months back? One of our shortcomings is a constant st ruggle to remember. How difficult it can be sometimes to recall the name of the person you need to meet in an hour, the important phone number your secretary just read you on the phone, or that very important item your wife told you not to forget to bring home this evening. But what if you had a magical device that would allow you to rewind reality and see exactly what happened? A few years back I began thinking about what could bring this dream closer to reality.
The features required to make such a device possible can be divided into three elements: first there is the hardware, which, as we shall soon see, actually presents the smaller set of problems. The software is saddled with a far more complicated series of hurdles, and finally there are social and perhaps legal issues that would inevi tably accompany the large-scale implementation of such a technology.
In terms of hardware, we would require one or two miniature high definition wide-angle cameras, which could take high definition (HD) video, that could be attached to our glasses or sunglasses. (They might look geeky at first but in a few years it might be possible to actually integrate them into the glasses similarly to MP3 and Spy eyewear glasses.) Microphones with the ability to locate the direction from which the sound originates would also be required. Other hardware add-ons could be eye-tracking equipment, which would register what you were looking at in any given moment (using equipment similar to the Stanford Poynter Project, for example), and a GPS receiver, which would inform you of where you where at any given time. Since most of our day is often spent indoors and current generation GPS receivers don’t work well indoors, new solutions would have to be found for this problem (such as inc reased sensitivity GPS receivers).
|
Perhaps the biggest hardware roadblock would be data flow and data storage. Using a wireless transmitter on the glasses to transmit real time HD quality video and audio would require a great deal of bandwidth, but more importantly it would consume a great deal of power, giving the recording glasses a very short operation time. Therefore, until more advanced batteries and power saving wireless broadband would become available, the best solution would probably be a wired connection, which brings up the second problem – storage. A DVD-quality video consumes about 4 MB/s, even with real time compression, we would still end up with around 1 MB/s if we would like to preserve a reasonable HD quality. If a full day (which would surely require more than one battery) were to be recorded, approximately 57 GB of storage space (1 MB/s for about 16 hours) would be required. Although this is within the limits of current day portable hard drives, it is still very expensive (about $25 per day on storage alone at today’s prices).
But hardware issues are slight in comparison to the problems on the software end. Let’s say that you have recorded a full month of your life and would like to find what you said to a friend in a meeting though you don’t recall when or even where it took place. You could of course start running the 480 hour video and look, but even fast forwarding, it probably would not be a very good use of your time. So how can you search a 1.7 Terabyte video for a few specific seconds? Well, currently there is no simple solution to this problem but there are a few technologies that if integrated might help. Voice recognition technology has been evolving for many years and recently has attained a pretty acceptable level. However it usually requires training and relatively ideal conditions (a microphone in front of your face and a fairly noiseless environment). For our purpose, the voice recognition software would need to be able to recognize what different people are saying without training and in less than ideal conditions (outdoors, in noisy crowded places, etc.). The software would also have to perform another trick, that is, recognize voices without long training sessions. Facial recognition, could be implemented to enhance this. By combining the information from the voice recognition and facial recognition softwares, it would be possible to analyze exactly who we were talking to and what was said. The eye tracking movement hardware could also provide important information in this respect. GPS could add another level by stating where the conversation took place and even what we were looking at at the time.
Even if all these hardware and software hurdles could be overcome to create a device capable of real time HD video recording and data analysis, socio-psychological and legal problems would inevitably crop up in our society in which privacy is a sacred value. Not to mention what what would happen if some unauthorized person somehow gained access to your life’s database!
Actually implementing the idea described above is still at least 10-20 years away, but, as described below, partial implementation is already underway and might bring actual benefits to all of us in the near future.
Bush’s Memex and a few life bits
|
The seeds of such a device were planted by Vannevar Bush, an American engineer who was involved in some of the political aspects of the scientific war effort in World War II, including the development of the atomic bomb and later the creation of the NSF. In an article published in the Atlantic Monthly Journal in 1945 entitled “As We May Think“, Bush discussed his ideas for the future, including a machine Bush dubbed the “memex”, which he described as follows: “A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory”.
Nearly half a century later, Bush’s conception of a device that can store many aspects of our lives is starting to become a genuine possibility due to several recent advancements in computer technology. Gordon Bell, a senior researcher at Microsoft’s Bay Area Research Center, began thinking about the memex in the late 1990’s. In 1999, Microsoft began MyLifeBits, an experimental project whose aim was to fulfill Bush’s concept by storing various aspects of Bell’s life on a searchable database. By scanning his books and old pictures, recording his phone conversations, meetings, and favorite radio stations, collecting his emails, viewed web pages, and IM conversations, and practically almost every other recordable aspect of Bell’s life, Bush’s memex dream started to come into fruition.
|
Since its start seven years ago, the MyLifeBits project was expanded to include real time video using a device Bell calls a Sensecam, basically a small webcam worn by Bell from time to time. Though a more advanced version is under development, the device has relatively low resolution and cannot record a full day of events nor does it include a GPS receiver or eye tracking movement technology. At present, most of the work in the project is focused on the software end. Multiple categorization, which is related to user interface, advanced storage, and search mechanisms, is an innovative approach that is meant to replace the hierarchical directory approach in which files exist in only one directory, with a system that would allow multiple categorization of each file, matching more closely the way people naturally think about organizing information. As an example, a picture of my family on a trip to New York would be categorized under both “family photos” and “my trips”. Although some commercial products already utilize this concept to some degree, it is still not natively supported by the OS itself. Dr. Dennis Quan, formelly of MIT, explored this concept in a paper he published in 2003.
|
Interview with Gordon Bell
To learn more about the MyLifeBits project, TFOT interviewed Microsoft’s Gordon Bell:
Q: When did you begin the MyLifeBits project and what was the motivation to start it?
A: The project started around the end of the last decade. The initial motivation was to scan my content and to get rid of all the paper in my life.
10/98: Bell to Raj Reddy, “It’s fine to scan and put my books on line. Don’t worry about copyright stuff. Microsoft has lots of lawyers.”
1/99: 1’st scans with “Digital Sender”, 4/99 “We” start to scan.
1/01: CACM “A Personal Digital Store” (16 GB; +2/yr). Began search for search engines, especially for email.
10/01: Jim & Roger join the project to explore the issue of using SQL as the platform.
11/02: New Scientist Article.
2/03: Techfest MyLifeBits demo.
10/03: SenseCam addition.
12/04: CARPE workshop founded by Gemmell.
2/05: Epiphany! Memex is a database & personal TP system.
2/05: NSF PIM Workshop@ UW.
Q: What were the different stages of the project and where does it stand now?
A: Here is a quick list of the main stages of the project:
1. Scan, search for a good search engine.
2. Start a serious project to build Memex using SQL.
3. Add all the desktop content we could: web pages, key strokes, mouse clicks, and phone calls.
4. Add real time video through the SenseCam.
5. Start of a Memex Research program with universities.
We are currently working on adding real time health and wellness data.
Q: How many people are working (or have worked) directly on the scanning etc.?
Q: How does the new search tool you developed for the project work?
A: We use various components from various tools. The initial was the text search in SQL. Today we use some of the search engines as part of the MSN Desktop search.
Q: What do you still see as the main problems of the project?
If you look at the overheads above, you’ll see a number of research projects that the project creates.Q:
Q: When are we all going to benefit from the MyLifeBits project, will some sort of commercial product come from your work anytime soon?
A: We are working with WinFS to get the functionality for a file/database provides and requires.
Q: Finally, as for your plans for the future, how do you see the project in 5 or even 10 years time?
A: Something beyond our current imagination, but that we treat as a wide range of use from repository for everything to a memory surrogate to something that becomes an assistant to be pro-active.
