Search and Discovery (cross posted from my BBC blog)

Search and Content Discovery

• Richard Titus
• 26 Sep 08, 2:00 PM

A few months ago, I gave one of the keynotes at our annual BBC Future Media & Technology conference.

I ended my speech, which ranged from an overview of the evolution of user interaction models on technology to cloud computing and the semantic web, with a picture of the Google search window...

...and the statement:

All this innovation, and yet this is the best we can currently do for content discovery: brute force text search. We have to do better if we want to evolve.

Okay, it was for dramatic effect, but I believed then and believe now that I was absolutely accurate.

Search is one of the darkest backwaters of technological and experience development (particularly on the internet.) Since then, I've been thinking a lot about how we, the BBC, can improve search on our site, and how we can drive innovation around search in general in the industry.

Earlier this month, there were a couple of really interesting launches in the world of search.

First, Yahoo! released Boss, which is a completely open, virtually limit-free search API. What's interesting about this is that it's a brilliant defensive move against Google's dominance.

Yahoo! is clearly Number Two, but since it's a marginally zero sum game in terms of monetisable search traffic, it needs a different way to take market share. "Embrace and extend", indeed.

This was followed almost immediately by Cuil, a Xoogle (ex-Google employee company - more about that later) which launched to much fanfare, and mostly collapsed into a mess of unmet audience expectations - always risky.

Frankly, I haven't played with it enough to make a decision, but it wasn't nearly as compelling as the rest of those mentioned in this post.

I was lucky enough to join Jane Weedon, our controller of business development, on a trip to Asia and to the USA to do some learning about small, young innovative companies and market trends.

In the realm of "oh my god, that looks like rocket science", we file Viewdle. Essentially, it's an image search engine with facial recognition software.

Born in the Ukraine out of what I suspect was a largely military development effort, the technology is funded by Anthem, a SoCal VC and frankly, after a thirty minute demo, I was blown away.

See for yourself at reuters.viewdle.com/searchm. I'm keen to spend more time on this, and feel like there's an unknown number of ways to leverage this.

One of my favourite meetings from Asia was Naver, the Korean search giant owned by the largest online gaming portal in Korea (another interesting space for a blog).

With near 80% market share in Korea (Google has less than 4%!), 16m people visit Naver every day. They have managed to capture and data cache the majority of Korean language content on the internet.

Now, to my non-Korean-speaking western eye, this is a confusing, hard-to-understand site, but there are some really keen innovations here:

• They mix different kinds of results into an answer, presenting only relevant ones
• Their scrap tool (sort of like social bookmarking à la digg or delicious) allows users to copy parts of one blog or site onto another, helping to grow the interconnectedness of the interweb and building relevance
• Behind Naver is an engine of editorial staff who review
• They have a Google Answers- or Yahoo! Knowledge-like offer which helps to identify new subjects and content to deep dive on
• Other than the aforementioned editorial staff, which is outsourced to low-cost centres like China, the company is run by a team of just over 80 people who are amazingly innovative and agile
• They also have JR Naver kids' search

Interestingly, US-based Mahalo lists Naver as its biggest inspiration. They have duplicated the Naver editorial model, but built it up into an amazing engine of content discovery and improvement.

Mahalo creates pages about selected subjetcs using its amazing editorial/ curation team which is distributed around the world . Their page curators, who come from all walks of life - professors, doctors, homemakers - create the pages for a nominal sum (under fifty quid) per page. It's a model similar to Wikipedia, but managed (ie, you have to demonstrate your skills and you are evaluated regularly to assess the quality of your work).

It is an interesting alternative to the approach taken by Daylife and others (including the BBC with its Topic Pages - previously blogged here) where pages are produced automatically using search queries to find and aggregate content. This is obviously cheaper and computers can find much more content than human editors ever could. But Mahalo's pages have a hand-built quality that can only be produced by skilled editors and well thought out workflows.

Co-founder and CEO (and a long-time friend of mine) Jason Calcanis talks about how this makes his content more "trusted"; which I think is a really interesting concept.

His new line, which I'll repeat here, is that trust is one of the most important currencies/assets in the digital future. Frankly, I'd put it up there with metadata.

Digital has a function of changing the nature and assets in the future. Attention, data and trust, rather than cash and inventory: brave new world, indeed.

Mahalo is a Sequoia investment. I was lucky enough, with some colleagues from the BBC and Sony, to attend a Sequoia open day in SF. It's essentially a beauty parade by the VC of their best and brightest (and most relevant) investments for larger strategic or VIP friends and family.

One of the most compelling things they showed us was SearchMe, which I was quite impressed by. Essentially, it's a combination of a new search engine (built by Xooglers) with a new, Flash-based interface.

Now, the interface borrows heavily from Apple's interaction pattern library and it's a bit clunky for browsing, but it is quite striking.

I find that it's also really good at predicting what I'm looking for, with a few exceptions. I see pieces of the old snap search engine (the creative director, Jason Fields, just joined us at the BBC) as well as X1 (long may it live - one of the most useful tools ever).

Frankly, however, it really shone when they loaded up the Searchme Ap on my iPhone. Oh MY GOD! WOW. Extremely compelling search on a phone (it replaces the internal search and I don't miss it a bit.)

I'm converted, though they need some slightly better browse mechanisms (see what Apple did in the newest version of iTunes.)

Cuil has some interesting visual metaphors as well: the blue type, minimalist and frankly ugly and not very usable Google UX seems to be crumbling!

I really think that the next two years will be defined by those of us who can really raise the efficiency of discovery (both targeted - ie, I know what I want, and browsing/snacking - ie, I'm looking for something stimulating).

When you marry solid data and indexing (everyone forgets that Google's code base is almost ten years old), useful new datapoints (facial recognition, behavioral targeting, historical precedent, trust, etc) with a compelling and useful user experience, we may see some changes in the market leadership of search.

Richard Titus is Acting Head of User Experience & Design for FM&T.