October 24, 2004 18:05 | Technology

Without you, am I nothing

Out of over 200 Gigs of data (not counting OS and application files), I would be hard pressed to find a single file that is not in one way or another associated to someone other than me.
(Someone may represent a person, a group, a company, my company, me, a webservice, a bot... any entity with at least one interface I can communicate with.)

Not a SINGLE one.

Current filesystems were designed in the computing age. Whether mainframe or standalone, the idea was you create a file and store it. Filename, date created, date modified.

In the communication age, things have to change. I've heard alot of talk of next-generation filesystems which are coming up but not nearly fast enough. I, and many people I know, are literally drowning in data. Drowning because we do not have the tools to efficiently manage it. The malaise and disorientation are evident. Subdued, subtle, yet evident. (Yes I am referring to YOU. And me. We've been acting "strange" these past few years haven't we?)

Before computer networking, the computer was an extension of two things: memory and processing. Fifteen or so years ago we gave computers eyes, ears and mouths. Over the last 10 years we've been learning how to use those eyes, ears and mouths. But without context (metadata!), we merely swim in an ocean of information we are hard pressed to do anything with. (Mostly because through these extra eyes ears and mouths, we are exposed to and generate far more data than with just our own set, in the "real world".)

We have these external memory & processing devices which we can now use to communicate, but we do not yet use them to help us manage that communication.

A simile.
A newborn child discovers he has this sense that perceives light. It spends a few weeks learning how to use this organ of sight; learns how to focus, move that focus around, etc. Great! Lotsa neat globs of color! But then something better happens. Context. That familiar smell, that familiar sound... the scent, heartbeat, voice of someone... oh! those are near when this particular set of color blobs are near! They must be connected!

The memory and processing has kicked in and created an association. (The single strongest association in all of our minds; the bedrock of our perception of reality: the parent. But I digress.)

We have learnt to use our externalized eyes and ears and mouths, and we've tried to manage the data they perceive by trying to store and process as much of it in our internal minds. But it is too much. And it will get "worse". We NEED to have the external systems that are our computers, help us process some of it. With context and associations. Some requiring our intervention, some done programatically. A balance of logic and fancy.

To those who say "but we are externalizing too much!" I say: Sure. We've been doing it since we invented written languages. We then proceeded to create technologies to manage the effects of written language, which handle some of the information overflow for us. Hello, bookshelf and library index cards.

Now we have extended our senses, so we need to extend our processing of their perceptions.


In science fiction (notably in William Gibson's work) the concept was called a memory palace, a kind of hyper-filesystem that encompassed communication archives, bills, accounts, contacts, all contextually cross-referenced.

There is some work building towards machine understanding of context, notably the decades-old work on CYC (for Encyclopedic), an AI system that learns the 'rules of the universe' much as we do. Theoretically, it could know everything about vampires and also know that they are fictional, for instance.

The next step to managing data is a richer interface (involving eye tracking, gestures, and speech) - such as Mark Lucente has been working on at IBM for equally as long.

If we haven't reached the ends of computer processing power yet, we may see this within the decade. If, as increasingly seen, physics are against us, it may be something that cluster computers do, powering networked workstations or websites.

Heeheee heeee... dude... any argument starting with "in science fiction" is immediately made fun of. Sorry. Them's the rules.


Machine understanding? And you are smoking what exactly my friend?

I've heard pioneers of AI, who have struggled for 20+ years basically say "ferget it, not gonna happen". Machines cannot think. What they CAN do is help us store, communicate and retrieve information.

We've got all three working together now, but we need to rejig much of the system to make it more effective. Starting with the "store" part.