The Promise & Reality Of Mixing The Social Graph With Search Engines

Robert Scoble used the theme “let’s blame SEO” as a launching pad for a series of videos on how Facebook potentially could be a killer search engine — regardless of the fact he seems to have no clue that “social graph” or social networking mixing has been tried and abandoned with search. Having watched his videos, which have sparked much discussion, I’ll do some debunking, some educating for those who want more history of what’s been done in the area, plus I’ll swing around to that New York Times article that ascribes super-ranking powers to SEO. Plus, I’ll use the F-word along the way.

Robert’s excited about “social graph search,” which is the idea that if you know a network of people, you can use their connection to improve search results. It’s a “revolution” coming in search that will overtake all the major search engines, he says. Maybe, but it’s not like we haven’t heard this before. I’ll go through his arguments, but it really feels like this is more about getting attention to Robert’s videos, period.

Part 1 of Robert’s social graph video series starts off by telling us that there’s no way we’d have gotten to his videos from a search engine. That’s absurd. People write about what’s in video content all the time. Want to see the Lazy Sunday video? Oh, look — I found it number one on Google without Google needing to analyze the words inside the video.

That’s the real point that Robert’s trying to make, of course — that search engines typically don’t analyze all the words within a video in the way they read the words in a web page. Want to understand more about that? My Video Search Challenge Isn’t Speech Recognition, It’s Content Owner Management post from February explains this in more depth as well as why it really hasn’t been an issue. In particular to Robert’s argument, it’s because there are plenty of people who will reference what’s in the video content in more user friendly and search engine friendly HTML text. Read the rest of this entry »

Previously I’ve covered what I dubbed Search 3.0, how search engines have evolved toward blending vertical or specialized results into “regular” web listings. Today, the step beyond that: Search 4.0, how personal, social and human-edited data can be used to refine search results.

The Search Evolution So Far

Before going ahead, let me summarize what I covered in my past article, in terms of how search engines have changed over time to create and rank the results you get when doing a search:

• Search 1.0 (1996): Pages ranked using “on-the-page” criteria
• Search 2.0 (1998): Pages ranked using “off-the-page” criteria
• Search 3.0 (2007): Vertical search results blended into regular search results

The evolution above is not perfect. For one thing, some “Search 3.0″ blending started to happen years before 2007. It’s just that in 2007, I felt all the major search engines made the leap into Search 3.0 in a significant way.

As for Search 2.0, looking at off-the-page criteria such as links, Google kickstarted that heavily in 1998. However, some link analysis happened before then, and all the major search engines probably didn’t get on board to using it more fully until 1999-2001. But the launch of Google in 1998 remains the benchmark year in my mind, for that particular change.

The evolution is also only applicable to crawler-based search engines, those that use automation to gather web pages, store copies of them and search through the compiled index to create listings for searches. Yahoo was a major player using human power before 1996 and continued this way for years. Indeed in 1999, a majority of major search engines were presenting human-powered results. This quickly changed as Google grew. Yahoo made its human results “secondary” to crawler-based ones (then provided by Google) in October 2002. Today, all the major US-based search engines depend on crawler-based results.

To cap off the caveats, the evolution above is not the only way search engines can evolve. That’s just how things have largely gone with US-based search engines, which in turn tend to also be the major search engines for most countries around the world. There are exceptions. For example, Naver is the dominant search engine in Korea — and there, listings are largely human generated.

Search 4.0: The Human Factor

Onward to Search 4.0! As I said in my opening, to me this is the move for search engines to make use of human data as part of their ranking systems. In particular, it means human data generated by you, by those you know or by human editors. Read the rest of this entry »

Last week’s search news was dominated by Microsoft’s announcement of Live Search Cashback. The idea is simple: You use search to find things to buy (68% of retail purchases start at a search engine, according to Microsoft). Folks who want to sell you things are more than willing to pay to get your attention (this is pretty much the model of all advertising). If you use Microsoft’s new Cashback service, those folks will pay you for buying their stuff, assuming of course you used Microsoft’s search as part of your buying process. The cash back items are shown in the right rail, mixed with traditional paid ads.

In essence, Microsoft has taken the affiliate model - where merchants pay channel partners for leads which turn into sales - and turned all of us into potential partners. If it sounds like a crass play to buy your search allegiance, well, it is. But Goto.com was crass too, and it turned into a multi-billion dollar market, the ultimate expression of which is Google. So before you judge it, it’s worth thinking about a bit more deeply.

There’s no doubt that with Cashback, Microsoft is attempting to disrupt the search marketplace. But there are only a few axes around which you can do that. One, you can disrupt the presentation of search. This is very hard to do, but it’s happened before, and will happen again. Secondly, you disrupt the business model of search. And third, you can disrupt how search is created (ie, the secret sauce of relevance). There are startups along every one of these axes of disruption. But with last week’s news, Microsoft is focusing on the second one (business model). Unless, that is, you read between the lines. That’s when we see the beginnings of disruption along lines one and three as well.

But to the specifics, and I’ll circle back to that idea at the end of this post.

First, Cashback is a well understood evolution of the same direct response model pioneered by Goto. Instead of paying for a click, or lead (CPC, CPL), advertisers in the Cashback program pay only when there’s an actual sale. This CPA (cost per action or acquisition) model is well trod (Google rolled out a limited version two years ago, and there are no shortage of companies playing in this space), but in this case the action is a sale, and this is the first time a major player is providing incentive directly to the person who is doing the searching. Read the rest of this entry »

After years of no real dramatic evolution in search, the third generation finally arrived. Google calls it Universal Search, and I’ve been tending to say “blended search” as a generic name for the change that’s now hit all the major search engines. But in doing the agenda for our upcoming SMX West conference, a better term for what’s going on finally clicked: Search 3.0. In this article, I’ll cover the why and what of Search 3.0, taking in Search 1.0 and 2.0 along the way and touch on how Search 4.0 — personal and social refinement — is on the way.

Search 1.0: Location, Frequency, & On-The-Page Ranking Criteria

Let’s go back to the good old days, around 1996, when some of the big names in web search were Lycos, Infoseek, Excite, WebCrawler, Open Text, Hotbot, and AltaVista. Yahoo was there, too — but it was the exception, the search engine that relied on humans to catalog the web, rather than the others that used “crawlers” to automatically build listings. Read the rest of this entry »