The Exploding Digital Universe

The latest posts are available via RSS
or delivered straight to your inbox by [Clicking Here]. New to RSS? Read an introduction to RSS.

Some light entertainment came my way over the weekend, although the fact that I class it as light entertainment reminds me that I should get out more…. I read the EMC sponsored update to IDC’s “The Diverse and Exploding Digital Universe” - a report on digital information.

The digital universe was apparently 2.25 x 1021bits (281 exabytes or 281 billion gigabytes) last year, with faster that predicted growth due to digital cameras and digital TVs. That is mind-bogglingly huge. IDC says the amount of information exceeded the amount of storage for the first time last year. This may be one of those landmark moments, like Internet traffic exceeding PSTN (telephone) traffic in the mid-90s, ushering in the on-line age.

If we are creating more than we store, this points to more and more transitory data in the digital universe. IDC predicts almost half of the worlds data will be transitory in 3 years time - created, then lost again. I suspect that increased streaming video services (like Qik) account for some of this, but the fact remains that we are moving from a create-forward-store universe to a create-stream-disgard one, and that is a big shift. It seems that even the digital universe itself is suffering from information overload.

The media, entertainment and communications industry is the digital giant, accounting for a disproportionate amount of data in the world. A single one hour HDTV program dwarfs a lifetime’s knowledge written into a text-based wiki.

Handheld/mobile devices are becoming a larger and larger part of the universe, with cheaper and larger flash memory technology, we are able to carry larger parts of the digital universe around with us. However, more and more ubiquitous Internet access on mobile devices also gives us access to information when we are busy being road-warriors.

External hard disk drive sales grew faster than expected, so either we have more data than will fit on built-in drives (which makes sense with more laptops around, with harder to upgrade disks), or we are backing up our data more. Hard disk crashes seem to account for more and more of the productivity glitches that people recount to me. A good data recovery strategy really is part of a productivity plan, even if it doesn’t feel like it.

A quote from Kurt Vonnegut, via David Terrar, “Life happens too fast for you ever to think about it. If you could just persuade people of this, but they insist on amassing information.”

Many businesses do have data retention requirements, with all of the issues that go with that. What about us as individuals? We do need to retain some data, e.g. for tax and other legal reasons, but might we benefit from a bit more archiving? Digital clutter can be just as bad as physical clutter. How does your desktop look? How long does it take to find the information you really need?

Most information used to evaporate: conversations were forgotten, contacts were lost and memories of events faded - unless it was written of course (one way to never loose another thought again). Technology now enables us to capture that information, potentially forever. The problem used to be information scarcity, now it is informatio overload.

The economics of information have flipped and the rules for managing it must change accordingly. Delete more, worry less?