This is the first article of a 3-part feature about what I learned through experience about real-time search, as the London Riots unfolded. Ultimately, speaking as a "search expert", I found following a mixture of users and media types on Twitter on a small selection of keywords was more effective than a crawler-based search engine. What I also learned from the experience was that, despite occasionally misleading content, there were enough collaborator voices within the stream who were requesting fact checks and improvising stories in real time - that the potential for better information existed moreso than the potential for misuse.
My quest to keep track of how the riots were escalating became an investigation into getting the best, most accurate information from social media. The insight I got from my experience was that all I need do to improve the quality of content and sources out there is to actively participate in organizing the news.
Go to The Winchester
As a native to London, and an expat living in the U.S., the riots of the past three days have had me worrying for the safety of friends and family. Without access to UK TV stations, I needed a workaround to get the latest news. Thankfully, I was able to get in touch with them via desktop and mobile before the trouble escalated on monday. First contact, via Facebook IM was, "everything is ok, but rumors of a mob heading towards Clapham Junction", meant that local workplaces had advised everyone to head home early. Second contact was with friends and family an hour later by mobile phone.
Turns out any fears were unfounded because they were in somewhere safe and familiar. The pub. Of course! A very british response. Keep calm and carry on. I'd even joked on Facebook about it not being a great day for the pub.
For a second, I glowed with national pride to hear that my best friend and my sister, in light of a possible emergency, had essentially resorted to the same plan as from Shaun of the Dead: see YouTube clip to go to the pub.
Shaun: Take car. Go to mum's. Kill Phil, grab Liz, go to the Winchester, have a nice cold pint, and wait for all of this to blow over. How's that for a slice of fried gold?
However, "the plan" rapidly unfolded within an hour. Clearly, events had got so close to home that mainstream channels like the BBC live feed and now, even Google (without the deal with Twitter), would be too slow to publish the latest news (Let alone the fact that UK TV channels were not able to broadcast across the pond). Meanwhile, whilst I was brainstorming where to look for news sources, hyperlocal hashtags were emerging on Twitter and friends were tweeting out hashtags for me to follow.
From New York, perturbing news of #londonriots was breaking on Twitter. What is more hashtags were starting to appear for local areas. #Hackney was going crazy. Soon #Clapham was appearing. Another call to my friend in London who (now exited the pub), described the scenes at Clapham Junction as, "Apocalyptic. Kids just running around with scarves around their faces", piling bags of loot into cars and vans.
From New York Skyscraper to Streets of London in One Hour
Following at least 5 different hashtags, focused on the boroughs of London in which I used to live, updates were streaming in - a lot of them helpful. Soon the rate of updates across different local hashtags was dizzying. Eventually a problem of repetition and misleading reports developed amid the stream of dismayed reactions to riots taking place.
New filters needed to be applied. The following steps were useful:
1. Create a list of what might look like eyewitness reports. Among the stream of tweets on the #londonriots hashtags were eyewitness accounts. Rather than following directly, I added them to a list - in the hope of building a clearer picture of tweets.
2. Lookout for Twitpic, Mobypicture URLs shared as pictures and videos on the ground, in the hope that they may provide more reliable eye-witness accounts. Searching hashtags of camera and photo sharing apps such as #instagram and #hipstamatic also helped discover eye-witness perspectives. Real-time photo search sites such as TwitCaps were useful, although Nachofoto totally failed in this instance. Later in the day, pictures started to surface on FlickR too.
3. Track real-time social story sites, such as Storify and Storyful. These tools also proved useful both as a way of documenting the most reliable sources of information and as a means of discovering more reliable sources. Google+ was also quite a useful tool account to save links as a story develops.
4. Subscribe to a keyword search feed on YouTube. Videos also started to emerge on YouTube within only a few hours, with video footage of the looting being shot from the street or second floor apartments. Creating a playlist of videos as they popped up meant that all of them could be easily shared.
Within about an hour or less, from New York, searching all these realtime channels had provided me a view from the streets of their neighborhood. Thinking about it for a second, that was amazingly fast!
However, finding this video of a reporter on the streets of Clapham Junction and Northcote Rd, documenting the looting, put all other news reports into perspective. The situation was serious.
As night set in, trouble started to spread to other parts of London and problems at current hotspots intensified, so the need for news increased. Racing home from work, I was checking my Twitter streams for blog posts, photos and video. Between the 30 minutes to get home from the office, I had found out a shop, well known to my family over the last 20 years at been set ablaze. Also, our local 'big box' store, Debenhams, had been completely ransacked, and there were rumors that it might be burning. Luckily, other Twitter users in Clapham were monitoring the feeds confirming and denying different facts cited among the messages streamed over the local hashtags. This unnerving video from Woolwich showed how severe the crisis could become in other areas:
The situation was worsening. The photo below is of a shop where my sister used to spend her summer jobs working in, burning down.
Another way to get news, without a TV, was to use video skype with friends and have them point their webcam at their TV broadcasting local channels (in this case the BBC). That was useful for about an hour. Realizing how the information from there was verified, it occurred to me to screenshot images and send them out via my own social networks - in the hope of providing verified news for other people. It quickly became obvious that to get better information from social networks, I would have to be a better source of info myself.
By about midnight, GMT, there were calls on Twitter to not retweet unverified rumors. Live blogs from the Guardian, BBC and the Telegraph helped to temper the rumors, whilst footage of retaliations surfaced on YouTube:
The need to start relying on Live blogs reinforced the fact that, to get the most reliable news, I had to find the best curator.
Whilst a backchannel remained open for me to chat with friends and family on Facebook and Skype, the real news was breaking on Twitter and supported by eyewitness accounts on YouTube. The lesson for me was that, in order to identify the best sources, I had to become a good listener myself. In not a dissimilar way that Blackberry Messenger (BBM) seemed to be the channel on which to orchestrate the riots, Twitter was the channel for an emerging civilian response. To tap into what that response, I had to be willing to curate the news too, and actively listen myself.
By engaging in the act of 'listening' to all the noise on Twitter and it's ecosystem of apps, new sources, story leads and apps, emerged all over the place.
However, from my Twitter list, certain users really took to curating the news and, more importantly, curating the ways that people could find it. Mike Butcher, editor of TechCrunch UK (and technology ambassador to London Mayor, Boris Johnson), did a sterling job of publishing his own coverage of the riots and also raising the visibility of important hashtags to follow.
Through Mike's Twitter account, I became aware of other iniatives that were going on. Reports of communities taking to the streets to defend their areas emerged, as did #riotcleanup, a hashtag upon which citizens were mobilizing to cleanup the destruction. A counter story was starting to emerge - the mobilization of a civilian response.
So, How Do You Search In An Emergency Again?
I still think it is pretty incredible that social media and smartphones with high speed data plans have become so ubiquitous that I was able to use the internet to get up-to-the minute information and eye witness news of the riots from the very streets of my hometown, directly to my desk in New York. Furthermore, the response was so fast that via SMS I was even able to provide or confirm news for my friends and family, in a timely manner that they could possibly respond to.
All of this was done via social networks rather than a crawler based search engine.
Had Google Realtime even existed, it's crawler based approach would probably have provided no real benefit either, because, more useful than simply aggregating the voices, was the process of participating in the conversation. More important than the role of aggregation from the platform, were the users who went to the effort to highlight breakout trends within the conversation and confirm rumours.
Search through social media, in this instance became more of a collaborative experience, not unlike the "human flesh search engine" phenomenon, found in China. Rather than using crawlers to find information published online, real-time search was at it's most effective when there were more users resolving little problems. Furthermore, this rapid response became faster the more hands were on deck.
Karma seems to be "King" in the realm of realtime search. So, to get the news you need in an emergency, then consider being a conduit for the information search for everybody and aim to make all the right channels easier to find. Below are some of the ways that people started to organize themselves on Twitter:
1. Real-time maps emerged on Google Apps, which mapped #londonriots tweets to areas of London and enable users to plot flashpoints via Twitter.
2. Dedicated accounts for #riotcleanups were created for different cities and followed each other.
3. Follow recommendations were made for individuals and for hashtags.
4. Content on Storify was ramping up with more citizens curating their own longer reports to be shared on Twitter. Another useful site which emerged was Russia Today - a sort of broadsheet style newspaper, that focussed on "real-time" news.
5. Also useful, were Ultra Knowledge, incidentally the team behind the new Search Engine Watch, also made a web tool which automatically curates all the activity on trending hashtags into a set of sources and different media types (pictures & video) shared on Twitter - making it even easier to cut through noise and clutter.
So, to the key to searching during an emergency comes down to searching for, identifying and collaborating with the social network users who are organizing a response.