16th January 2012

The recipe for the work of a news organisation in this internet age sounds easy. Gathering the news is a problem which has been solved more-or-less since the AFP and other news agencies were founded in the early 19th century. Plus or minus the occasional investigative journalism and the obligatory opinion piece, the value added of a newsroom is to carefully comb through the insane volume of information, clean it up (rewrite it, summarize it, summarize it some more, find a catchy title), organise it, add a bit of explanatory text, and sell the packaged result.

In other words: curation for the museum of now.

The only problem, everyone seems to agree, is to figure out how to make money out of it when the web provides everyone both with a free way to replicate that careful product of editorial brilliance (no hiding behind a costly piece of printed rag any more) and immediate access to a gazillion competitors, including the source itself, the wire services. And so every news organisation scrambles to earn its due, and find a way to make money out of their obviously valuable editorial process.

The problem is, the kids are not telling the curators that their museography isn't interactive enough, or that the cardboard signs next to the artefacts aren't glossy enough. What they appear to be saying is “Sorry, we have access to the whole catalog now, and we have our own ways of navigating it. Find another job.”

Unfortunately, all is not well, because there are killer bubbles out there.

Bubbles? Yes, bubbles. Dangerous bubbles. Filter bubbles. In Eli Pariser's now famous analysis, our society is not only casting away the top-down editorial gatekeepers of yore, we are replacing them by a mix of like-minded friends and well-meaning algorithms which shows us only the information we are supposed to find interesting. Great. Except that by automatically filtering away all the stuff we are unlikely to find interesting, the algorithms take away the chance to decide that something which may not usually be interesting is worthy of our attention anyway. The filter bubble keeps everything "boring" (or indeed, everything we may disagree with) in other bubbles far, far away. In other words, the filter bubbles are thought to be dangerous because they transform a variety of viewpoints (good for democracy) into a segregated myriad of sub-cultures and mindsets (not so good for democracy).

This is a serious allegation against the “democratising effect of the Internet”. As a proponent of an open, democratic public discourse, I do see in the “Filter Bubbles” a good explanation of a hunch I'd had for a long time - what if hyper-personalisation of information in the digital age was too much, too soon? Am I not seeing Google's Search results slowly approaching an uncanny valley of knowing so much about me, of being so good at giving me what I am looking for, that serendipitous discoveries have become an a near-extinct species? Some have argued eloquently that the internet does not create filter bubbles any more than traditional media, but there is a general agreement that mass-automation of the personalisation of media is likely to make things worse, unless we keep a stern eye on it all.

And that, ladies and gents, is the sound of a thousand newsrooms breathing a sigh of relief. Livelihood assured: without an editor carefully selecting the front page, we may be missing important things that any self-respecting citizen should know.

That's the happy ending of the first episode. And as we all know, the second episode is kind of dark and gloomy and tortured and full of weird, incestuous plot twists. But I digress. Maybe.

The fact is that our industry seems to have acknowledged a problem (filter bubbles) and jumped on the first, oh-so-convenient solution: let us acknowledge the usefulness of social media as a way to determine what is interesting, while reinforcing the importance of editors as heralds of what is important.

And that's where we get our current landscape of news. The fully editorial (your typical newspaper), the social and personalised (the filter-bubble land of Facebook, Twitter and the buddingly social Google), and in between, a lot of attempts at social or personalised news.

Good. But I think we may be making two important mistakes here.

One is the relative weight given to "important" and "interesting". Look at most online news media today and you will find a small user-curated box somewhere down the right column — a selection based on what gets viewed, or shared, most often. It is a crude but decent measure of popularity: “if most people read this, it's probably interesting to the greater number, so let's make it more prominent”.

News Curation Surface

Comparing surface of crowd-curated news on the BBC, NYT and Guardian FB app

Among "serious" news outlets, it seems only the Guardian has dared give the popular content a lot of prime space: in their facebook presence (pictured above alongside the BBC and NYT) they have given center stage to a selection of the most popular articles on that platform. From what I remember was said at a recent UX event at the Guardian presenting said Facebook app, this choice may well be one of the reason behind the fantastic success of the Guardian's app, especially with the younger demographic which is too often deemed uninterested in news.

And why shouldn't it be successful? According to many of the studies about news-related attitudes and behaviour I've had a chance to look at, the need to "know what is going on in the world" tends to be associated with an anxiety to "complete the news", whereas the desire to be entertained or discover is much more casual and frequent. Sounds like an opportunity to test how an online news site reversing the relative weight of important/interesting, or finding new ways to mix or separate the two, would fare.

The second mistake I think we are making is in the terms of the truce between crowd/social and editorial. As mentioned earlier, the deal is that editorial in charge of the Important, and algorithms in charge of interpreting the people's behaviour and extracting the Interesting. Good fences make good neighbors, and all that.

But wait a moment. If we are going through a decade which has shown us that the crowd can be just as good as, if not better, than the pros at reporting the news or unearthing the whimsical, then why wouldn't the "wisdom of the crowd" apply just as well to curating the important news?

I therefore wonder if anyone has properly experimented with the crowd-sourcing of the curation of "important" news. And I suspect noone has: implicit curation (most read or most shared articles) only helps determine what is popular, whereas explicit curation (reviews, ratings or bookmarks) tends to answer so many questions at the same time (Is it "good"? Well written? Do I agree with the point of view? Is it informative? Is it funny? etc.) that it cannot remotely be used as a measure of what people consider important. News aggregator such as Google News seem to measure the importance of a story by how many sources have put it in their top news, but that doesn't solve how the stories should be put there in the first place.

Surely, there has to be a better way, and I haven't yet found anyone doing it right, or doing it at all. It may be a new way of looking at news consumption behavior and automatically detecting what people consider "serious reading": is it what they read first? Last? In depth or at a glance?

Or it may be an explicit user interaction as simple as asking them whether they found the piece interesting, and whether they deem it to be important. And see how they behave over a period of time.

You could imagine that the two metrics are then used in different ways in influencing my personalised news, and everyone else's: what I find interesting is used mostly for my personalisation (and a little for everyone else) whereas what I find important goes into a big pot which an editor can still control – no need for this to be entirely automated.

Nobody really wants a world of news filter bubbles. But the belief that no algorithm can ever do as good a job of selecting the important news as a human editor may well be the kind of luddite delusion we've seen painfully disproved time and time again, and should not be the only alternative.