by moleitau (cc)

It has taken me a couple of days to write this post, because my brain is still crunching on its contents. It touches on so many different areas of the technology and business areas that I am passionate about, that I’ve had to give up covering them all in one post.

The historic destiny of data (and it’s big parent, knowledge) has been to be locked up; the constant struggle has been to set it free. From the formation of  the university to the API-rich Web 2.0 world, people have postulated ways to make it more available. Of course, not all data is going to be ‘free’ (as in beer), but much of it should at least be accessible and usable (‘free’ as in freedom). The trouble is that much of today’s digital information is trapped in non-portable and hard to process formats.

It’s a vision at the heart of the semantic web, championed by Sir Tim Berners-Lee – particularly under the banner of Web Science, down the M3 from where I am writing, at the University of Southampton (this TED video is a useful primer).

Time to join some dots. Prime Minister Gordon Brown appointed Sir Tim to work on opening up government data, and last month Sir Tim talked to the Cabinet about a goal of delivering a single online access point to Government information, similar to the one introduced by the Obama administration in the US, and to what the Australian government has been doing.

Back in August I interrupted the weekend to visit Google’s London office and see a bunch of young folks, with a bit of help from some not-so-young ones, hacking together web-based services under the banner of Young Rewired State. It really was inspiring stuff. There were some familiar faces (including @grantbell) and plenty of new ones, all working on an impressive list of apps, adding to the list of Rewired state projects.

At the end of the weekend, an impressive list of judges (@craigelder @jobsworth @BenHammersley @helenmilner @marxculture and @danielheaf) judged the apps, and there are some good blog posts on what went off, as well as my usual collection of photos:

As a side note, I like the idea of hack days. Development sprints where you focus on one thing are a great thing – every business should have “hack days” – pick some key problems then get everyone together to spend a day working on fixing them. Hack days aren’t just for coders.

Back to that data, or more specifically the mountains of anonymised data that the UK government collects every year. Putting aside the usual “we paid to collect it, so we should have it” argument, so often espoused, there is a much better reason for putting all of that data “out there” – or there is to me at least. There are a thousand creative and useful things that could be done with it, most of which fall under the umbrella of  micro-business or hobbyist. These are things that the government, and most businesses, could never justify funding, but which a sea of enthusiastic developers could make happen – tackling all sorts of problems along the way.

Enter data.gov.uk. A surprise email last week meant, literally a few hours later, I was huddled around a projector looking at one of the most exciting things I’ve seen on the Internet in a very long-time. A very long time. With a very exciting team of people too. The previous day, via the Digital Engagement blog, the Cabinet Office issued a call for help:

From today we are inviting developers to show government how to get the future public data site right – how to find and use public sector information.

The developer community through initiatives such as Show Us a Better Way, the Power of Information Taskforce, MySociety and Rewired State have consistently demonstrated their eagerness and abilities to “Code a Better Country“.  You have given us evidence and examples to help drive this forward within government.

The new site will provide a way to access (eventually and hopefully) most of the UK government’s published data. There are already over a thousand datasets, in differing levels of accessibility from CSVs, to SPARQL end points, as well as XML and JSON. If those sound like alphabet soup, don’t worry, just be as excited as the developers who know what they mean.

The potential result is a wave of new applications, based on government data, that could do a wealth of things, from relating performance with class sizes in your local school to understanding how your local farming community is faring. Data.gov.uk is a very non-trivial project, and there is a long way to go, but what I was a very promising start. The early developer community is already very active, even though the site won’t really be in beta until the end of the year. As Harry Metcalfe puts it, the wraps are off.

There are four aims behind the project and opening up the data: 1) Aid transparency and accountability. 2) Empower citizens to drive public service reform. 3) Unlock the social and economic value in the data. 4) Stimulate the UK’s digital economy, with regard to technology and research in the web domain.

Certainly there are big commercial uses of the data, and I am sure big business will muscle in on the act, but underneath that, what a great opportunity to give a new generation of digital talent something to dig their teeth into. My hope is that the data will provide a platform for an ecosystem of businesses and micro-businesses, as well as non-profit organisations, to create value for UK Plc, both for the public good and for economic good. It will also be a proving ground for a new generation of geeks who can work with massive datasets and produce insights from them. Exactly the kind of folks the knowledge-based business of the future will need.

Related Posts

• Open Data Opens Up Gov
• Tim Berners-Lee @ The Innovation Edge
• Gordon Brown Announces “Second Generation” Government
• Future of The Web – Part I – A History

Last night NESTA played host to Tim Berners-Lee, with a talk under the title “Future of the Web“, followed by a Q&A and panel discussion. I’ll come back to the talk, because I want to start somewhere else first: in the past. The history of the web may well provide the best insight into its future. During the Q&A, Bill Thompson, asked a question that reminded me of a meeting I missed a few decades ago.

My brain is great at holding complex, interconnected models, pulling up random associated facts and remembering faces, but my episodic memory is awful. Seriously, ask me what I had for breakfast. I have no idea. May be you have no idea too? Ok, you don’t know what I had for breakfast, but you know what I am driving at. So, when my brain reminds me of things from three decades ago, I pay attention. 

A long, long time ago, I opened a very large package. A very, very large package. In one part was something that looked like a typewriter, except for a sad lack of space to put paper or ink in it. In the other was something that looked like a television, but it was completely unable to receive even BBC1 (in the days when we had less than 4 TV channels in the UK). My dad told me it was going to change the world, and that I should figure out how to use it. My dad was smart like that. He still is.

So, I got to work. I fell in love with that box. Most people that know me will tell you I am still in love with its offspring and distant relatives today. However, there is something in particular that captivated me about it. It wasn’t the ability to type in words and get it to do things – although I did use that capability a lot, and made some very good money in the process, thank you. No, the magic moment for me was when I got another, inauspicious beige box, called a MODEM. A clever box of tricks that allowed the computer to abuse a BT telephone line to talk to other computers.  That might have seemed a little pointless to most around me back then, but connecting computers together was rocket science. And everyone knows that boys love rocket science.

I could dial into something called a PAD, and from there, I could hop to another pad (landing pad, get it?) and so on, until I arrived at a big computer on the other side of the world. This was in the days when international phone calls were inconceivably expensive, and when the only americans I had seen were in movies. In fact, even the local phone calls to those PADs resulted in a £1,000 phone bill.

Now, strictly speaking, I shouldn’t have been on that system. In fact, I’d been a little creative in getting the numbers and codes to access it. That sort of creativity wasn’t illegal back then, and truth be told, the administrators knew that I was there and seemed quite ok about it. What harm was a kid like me going to do? They sent me nice messages, and we got on ok.

Later, they did start to get a little fussy. So I, and many others like me, started to write programs for our little machines to do some of the things that those big machines did. And much more too. Some of us got a couple of phone lines and  MODEMs. We made our systems available for others to dial in to, creating places where they could leave messages for each other and exchange programs. A kind of electronic bulletin board system, or BBS for short. Those phone lines ran at about 1/1000 the speed of the first version of bluetooth – if you think moving pictures off of your phone is slow, you’ll know why there weren’t any pictures at all.

Now, of course, we could have got in our cars and met up, but many of us didn’t have cars. To be frank, many of us weren’t the kind to strike up a conversation with a stranger, or to go out and find people to get to know. We weren’t in the social ‘in crowd’. We didn’t know it yet, but we were geeks. Real geeks.

We exchanged ideas, we explored new ways of using these machines and were generally pretty excited about what we found to do with them. So excited, that we started to meet up face to face, to talk about it all and to swap programs. Then we started to connect our machines together, so that our conversations weren’t isolated in little islands, but flowed like rivers around the world. Mostly, all of this all happened for free, powered by volunteers.

A little while later, I got dragged back to that world of PADs and the systems that belonged to the big people. This time, I was on the inside, as a student and then a lecturer. Those systems were connected together too. A sort of network of networks, or inter-net. Some years later, that was where I first came across hypertext (I’ve written about that before). One of my friends even wrote a program that bypassed the electronic message system and let you send messages directly to another user’s terminal. This was all before the thing that we call the worldwide web.

There’s something about that story. It was all about the people, not about the systems. Together, we steered what happened with the technology, both consciously and unconsciously. A few decades from now, this system I have in front of me, and the Internet it is connected to right now, will seem as alien as that first PC and those early bulletin boards seem now.

Last night Charlie Leadbeater drew some parallels between Internet users and the Levelers, a 17th Century pseudo-political group, who had an agreement to support freedom of the people.  I think those early Internet pioneers, the ones who sweated over the fuzzball routers and the ones who ran the BBSs were levelers at heart – creating something much bigger than themselves. Working with high ideals, connecting people in an attempt to build knowledge. It didn’t always go according to those ideals, and some things failed. Charlie said levelers failure was caused by the lack of an economic model. Well, the Internet has an economic model, all be it a very complex one. However, whatever it becomes, it is all about the people.

To be continued…

Related Posts

• Future of The Web – Part II – The Future
• Communicating Feelings on the Future of the Web
• Tim Berners-Lee @ The Innovation Edge
• Set our Data Free and Create a (Digital) Economy
• Gordon Brown at NESTA – The Innovation Edge

My first encounter with Sir Tim’s work was back in 1990. I was at Kent University at the time, and got involved in the trial of something called “Hypertext”. Quite frankly, I didn’t get it. A page of text where you could click on a word and another page of text would come up. What was anyone going to do with something like that? I took the pay and went back to circuit boards and DJing at the college radio station. Two years later I was back working on Internet technologies. The experience has made me much more thoughtful when I encounter new technologies!

As Sir Tim recounted the foundations of the web, it was curious how accidental and casual the whole thing sounded. The project was a ‘back-room’ effort, carried out during a lull in the work on the Cern accelerator. As Tim put it, giving staff a long leash, giving them space, is where innovation comes from. It brought to mind Google’s 20% time. You could hear the pens of the managing directors in the audience scribbling furiously.

The point of innovation is that you don’t know the end product before you start, sometimes you don’t even know the problem. It is a big risk, and that makes it tricky for a traditional management mindset. The web (or rather hypertext) was driven by the challenge of dealing with lots of documents and having to get others up to speed on them quickly. Necessity is the mother of Invention, which is a theme that came back later during the conference.

He described blogs as a social machine, which is an interesting metaphor – I guess we are still cogs in a machine, even in the world of social media. I have seen what this social machine can create form the huge diversity that it brings together. For me, two stand out quotes from the talk:

“The people doing the really interesting things tended to fall between two stools”

“The web really has to be thought of not as a system of connections between computers, or even as links between web pages, but really as humanity connected.”

The first made me think of the Medici effect, and is the reason for The Web Science Initiative, which sounds very interesting. The second was the final take away for me: The future of the web, and of innovation, is in individuals working in collaboration. There again, that is how all the greatest achievements have come about.

Related Posts

• Set our Data Free and Create a (Digital) Economy
• Future of The Web – Part I – A History
• Future of The Web – Part II – The Future
• Communicating Feelings on the Future of the Web
• Gordon Brown at NESTA – The Innovation Edge