Summary Response Week 3: Continuous Capture

CPSC 601.25 Week 3 Response

Papers

In this response I will discuss two papers regarding 'continuos capture' technologies, the different approaches they take to the problem and the different results they achieve.

Always On

Both the papers reviewed this week have a common theme, the continuos capture and storage of data. As hard drives have grown larger and cameras, sensors and scanners have grown cheaper, many people have become interested in capturing all the data we interact and storing it outside of our brain. This 'second brain' idea dates back to the very early part of computing and AI research and even science fiction. But like some ideas in science fiction, the journey to reality is difficult. The idea of constantly recording cameras and audio recordings becomes bogged down in the issues of privacy, security and social discomfort. But when the scope of the idea is reduced, and the technologies is used to address the special needs of patients with memory difficulties, the results are very good.

Lost In Scope

The first paper reviewed work done at Microsoft to create a database of everything, a system called LifeBits. This system incorporates data from a variety of sources, scanned images, continuously running cameras, audio recorders, fitness sensors and others. The developers of the system then attempt to store this data centrally, in a SQL Server database of all things, and then built software for searching and exploring it. As the paper is somewhat older, based on work done in 2005 we can consider how some of what the developers imagined has played out. The work can be seen as falling between two chairs, what is valuable is done, but not in the way the developers imagine, and what is not is simply ignored. The developers imagine a system that tracks the metadata of many of yours files and makes it easy to search them by specifying things like "within this period of time" or "related to this person". This is used every day by millions of users of Gmail and Spotlight (on OS X). Modern systems index and support the natural queries that people would actually like to use, but the data is living in the cloud rather then on a users LifeBit system. The developers also imagine a continuously flow of pictures and interactions over a timeline, this can be seen by any user of Facebook. Even versioning (which they imagine to be handled by Windows) is actually handled well by Dropbox. Rather then a monolithic central repository many of the useful bits of this system are handled organically by different providers and software solutions. The parts of their system that fly in the face of social norms, like wearing a permanent always on camera while you go about your day, is completely ignored by most people.

Helping People

Contrasted with the first paper, which focused too much on technical details that were outdated only a few years after it published, the second paper was fantastic. Developed by researchers in Cambridge at Microsoft Research, the SenseCam technology was developed especially to help people with memory difficulties. There SenseCam technology was designed to capture still images periodically while being worn and to respond especially to changes, such as in temperature, light and audio signals. A user would wear the cameras during a special outing or event and then repeatedly review the images and associated them with events. This is groundbreaking in two ways, by capturing visual images the user is likely to associate with this images better then if they were using only text and by making the use and review of the system very easy and frictionless. When studied with an actual user, the results were very good. We can imagine extending this work to allow sensors to monitor important "moments" in the brain, and firing them when a person has made a decision. Or even improving the review system to work with some of the psychological research on "forgetting". The issue faced in the earlier paper about people being uncomfortable about the camera is likely to be lessened, as the obvious utility to the person using it is so clear.