The Bridge: an experiment in bridging social networks and empowering users

As part of Simpleweb Playground, we’ve spent a short stint creating a tool that shows you everything your Facebook friends are posting, pretty much as they post it – an un-curated news feed.

It was the first step in attempting to look at today’s social networks in a new light, and ultimately to look at one possible way of giving back some control to the millions of social network users around the world.

Above all it was a really interesting challenge. We weren’t sure whether or how it would be possible, and we learnt plenty about techniques and technology neither of us had used much before (particularly scraping and websockets with node.js and react.js).

Behind the technology challenges were some big ideas – including the recent attention to how much (or little) control we have over our Facebook data and specifically a couple of ideas in this interview with former Facebook product manager Antonio Garcia Martinez.

Firstly, how unconnected our social network accounts currently are. My Facebook account is locked into Facebook. It can’t really post or listen to someone’s Twitter account. Which sounds obvious and inevitable, but is it?

“Imagine if you somehow went back in time and tried to create or architect a social network in the ’90s or the ‘80s. What would it look like? You’d…stipulate your own, user-controlled social-media file, which would basically be the data that you could pull off of Facebook’s platform, for example, right? That’s something that I would own and it would almost be like an email. Like, I’ve got my data and I can shop it around to whatever social network I wanted.”

The second idea is around how little control we have over the content we consume on social networks. Or more interestingly, how more and more people seem uncomfortable about that.

“There’s this notion of the algorithmic path. Way before Facebook, going back to Google, they’ve always claimed, “Look, we’re just intermediaries. The algorithm optimizes for a metric, whether it’s engagement, clicks, or whatever. We’re not responsible for what you see. At the end of the day, it’s you, the users, through your actions, that ultimately define what you see. It’s not really us.” I think we’re reaching a point where people are unwilling to write them that blank algorithmic check… The real issue is that people don’t assign moral agency to logic and algorithms.”

News as it happens

We weren’t aiming to architect some kind of new, open protocol – like email – for social networking. And we weren’t aiming to create a new social network that avoided ‘logic and algorithms’. But we did wonder if these would even solve the problems we face today, even if they were actually used.

Instead we had more humble objectives: a starting point to explore these ideas in a rough prototype. So, we thought it’d be interesting to see how easy or difficult it would be to collate our own Facebook news feed that:

1. could be combined with posts from other social networks (ultimately as a bridge between currently disjointed services), and
2. would bypass Facebook’s algorithms (ultimately giving the user more control over what they see)

The tool we managed to create just about ticks those boxes. It pulls in the posts that your Facebook friends are sending, including the ones Facebook excludes from your actual news feed. It wouldn’t be very difficult to add some user control over the feed – content filters, for example – and control of the number of friends to trawl. And it wouldn’t be very difficult to pull in tweets sent in each given time period and display them in our collated feed as well.

In some ways, getting to this point was easier than we expected.

Technical hurdles

Facebook still offers a completely javascript-free website that makes it relatively straightforward to load pages and read content from your own news feed in an automated way. And there are plenty of libraries that make this kind of scraping as painless as possible – in particular Simplecrawler – for discovering, queuing and fetching pages, and Cheerio for interpreting them.

Facebook also handles authentication in a pretty straightforward way so we were able to get our automated crawler acting as a signed in user.

Saying that, it took us nearly a day to crack getting the crawler acting as an authenticated user. The difficulty was partly getting Facebook to trust the requests our scripts sent to sign in, and partly passing the authentication cookie between two libraries.

The next challenge was getting a stream of posts that stays up-to-date. It didn’t take very long to be able to do a single trawl through our friends and pull in their posts from a given point in time. However, it took longer to get future requests pulling in only more recent posts. It took even longer to automate those updates so, with one request, a connection opens that triggers repeated trawls, each of which pings back new posts.

Finding the real solution

The biggest challenge was the reason this tool is just an interesting experiment, not a real solution for bridging networks and empowering users – at least just yet.

The tool relies solely on interpreting the markup on Facebook’s pages. It was fiddly, to say the least, to isolate and make sense of the bits of each page we’re interested in. And, more importantly, at any moment Facebook could change their markup and instantly break the tool.

What we’ve created so far is fundamentally brittle. And it only reads content. If you want to post or comment or react (i.e. if you actually want to network), you need to go to Facebook itself.

That said, as our experiment continues we’re looking to extend the tool – giving the user some control over how many and which friends to display as well as filters on which posts to include. We also want to add other social networks to the feed, starting with Twitter.

Although our project is called ‘The Bridge’, and is an investigation into bridging current forms of social networking, we’re starting to see it as a small philosophical cog in the start of a movement towards a new kind of social networking that might not look like anything we use today. Some of these ideas are explored further by entrepreneur Anil Dash in a recent piece on the ‘missing building blocks of the web’.

You can take a look at the Bridge project for yourself on Github. Don’t forget to stay tuned to the blog or sign-up to our newsletter to keep up-to-date on our progress.