The best way to prepare for the future is to look at — and learn from — the past. Danny demonstrated that as he took us through Search 1.0 and 2.0 as a framework by which we can think about Search 3.0 today and 4.0 tomorrow.

I’m going to take a different approach — let’s call it the next best way to prepare for the future. To get a sense for what search might look like in five to ten years, I suggest we probe far beyond that and work our way backwards. Join me on the path to ambient findability.

A phrase coined by Peter Morville and the topic of his fascinating book, ambient findability refers to the all-encompassing ability to be locatable or navigable.

I explored this topic in depth across three columns, ruminating on what it means for personalization, the potential downside, and select quotes from the book in the context of search marketing.

The Cliff Notes version is that ambient findability speaks to a world in which everything and everyone can be indexed and found anywhere at any time.

Work With Me Here

Let’s put aside all the challenges associated with reaching the point of pervasive computing that would activate ambient findability. Morville devotes the first couple chapters of his book to addressing these hurdles and, while they are not small leaps, they are certainly not insurmountable.

In fact, there are tangible examples of corporations harnessing the power of ambient findability today. One great example is the MyLifeBits project being conducted by Microsoft. I’ve written three columns about MyLifeBits focusing on its impact on personalization, ability to succeed, and application to search marketing.

In a nutshell, MyLifeBits is a memex (memory extender) created by digitizing and indexing an individual’s entire life — online/phone chats, media consumed, pictures of people/objects encountered, etc. – and, of course, making it all searchable.

Per Gordon Bell and Jim Gemmell, the brains behind MyLifeBits, the ultimate goal of the project is “complementary computing, where the computer understands human limitations and fills in the gaps.” In other words, once they’ve built a massive index of everything in a person’s life, they can create “a machine that can act like a personal assistant, anticipating its user’s needs.”

Back to The Here and Now

It’s anyone’s guess how long it will take until Morville’s dream of ambient findability is realized but I think we can all agree it’s not a matter of if, it’s a matter of when.

And it also seems assured that search will play a key role in helping us navigate a world in which everything and everyone can be found. The only question is whether search advertising, or more appropriately, query marketing, will be a sustainable revenue model in a world of ambient findability.

So what can those of us with a stake in search marketing do today to prepare for that eventuality?

I think the first step is to embrace Mr. Battelle’s words from his inaugural post on this site. We have to truly consider how flexible and powerful search can become when it is unhinged from the standard approach to which we’ve all become accustomed. And, while you’re at it, go fly a kite.

Aaron Goldman is VP, Marketing & Strategic Partnerships at Resolution Media, an Omnicom Media Group Company. Resolution Media helps marketers connect their brands to their audience through queries. Aaron blogs at FindResolution.com and GoodURLBadURL.com and can be reached at AGoldman@ResolutionMedia.com.

Robert’s excited about “social graph search,” which is the idea that if you know a network of people, you can use their connection to improve search results. It’s a “revolution” coming in search that will overtake all the major search engines, he says. Maybe, but it’s not like we haven’t heard this before. I’ll go through his arguments, but it really feels like this is more about getting attention to Robert’s videos, period.

Part 1 of Robert’s social graph video series starts off by telling us that there’s no way we’d have gotten to his videos from a search engine. That’s absurd. People write about what’s in video content all the time. Want to see the Lazy Sunday video? Oh, look — I found it number one on Google without Google needing to analyze the words inside the video.

That’s the real point that Robert’s trying to make, of course — that search engines typically don’t analyze all the words within a video in the way they read the words in a web page. Want to understand more about that? My Video Search Challenge Isn’t Speech Recognition, It’s Content Owner Management post from February explains this in more depth as well as why it really hasn’t been an issue. In particular to Robert’s argument, it’s because there are plenty of people who will reference what’s in the video content in more user friendly and search engine friendly HTML text.

The meat of his first part is to talk about three different types of search engines: crawlers (like Google), Techmeme and Mahalo to discuss how they are or are not “SEO resistant,” as if SEO is a bad thing — you know, that SEO equals spam.

SEO is not spam. It’s like saying email is spam. There’s email; there’s email marketing; there’s email spam. These are all different things. You want to better understand why SEO isn’t spam? Then read the posts below:

• Yes Virginia, SEO Is Rocket Science – Defending Search Engine Optimization Once Again
• Defending SEO, Yet Again!
• Why The SEO Folks Were Mad At You, Jason
• From My Inbox: More Defense Of SEO
• SEO: Real Skills That Can Protect Your Traffic

Want to be like Robert — and Jason Calacanis — and keep equating SEO with spam? Then f- off.

I don’t think I’ve ever used the F-word in any of my writing, and my apologies for being so crass. But I’ve had enough of people trying to advance their own personal agendas (Jason hoping someone will care about Mahalo; Robert hoping someone will watch his videos) on the back of an industry that is full of plenty of people who do good work.

Last week, I was part of a meeting at Google along with a number of notable SEOs, being asked about ways Google could be better. This group wasn’t pushing for Google to make it easier for them to spam the listings. A chief concern they had was how Google (along with other major search engines) continues to have difficulties identifying original source documents. You know — you publish your blog post, then some other site with more authority than you picks it up, and then that site gets the top ranking. SEOs are leading the charge to help site owners get a fix for this overall. But all people like Jason and Robert want to do is characterize them as evil comment spammers for their own personal gain.

Back to the video. Robert goes through how search engines make use of “on the page” factors, though he doesn’t call it that, and greatly simplifies the process. But yes, search engines look at the frequency and location of words on a page to determine if a page is relevant to that.

Robert then explains that PageRank is also used, using incorrect shorthand for link analysis that’s part of “off the page” factors that search engines use to rank pages — looking at the quality and the context of links to a page. My What Is Google PageRank? A Guide For Searchers & Webmasters post from April goes into more depth about what exactly PageRank is and how it is not the same as link analysis. Give it a read, Robert.

Next, we get the news that paid links are hard for Google to tell apart from “real” links. Actually, Robert says Google can’t tell the difference between them. In reality, it can easily identify many types of paid links. But not all of them.

Apparently, the SEO community feels it’s its “birthright” to stick paid links into pages. Actually, Robert — there’s disagreement about that within the SEO community, and the bigger audience that feels it has a birthright to do what it wants with paid links are the content owners themselves. SEOs aren’t selling the links; they’re buying the real estate that others are selling.

Robert then shifts gears to Techmeme and how, in his view, news won’t get on the site until someone starts to blog about it. Um, yeah — that’s part of the “meme” part of Techmeme. New stuff can (and does) hit the site pretty quickly, too, since important blogs catch things fast and talk about it. FYI, Q&A With Gabe Rivera, Creator Of Techmeme from me in January talks about Techmeme and how it works in more depth.

Techmeme is described as an SEO resistant site. Sure, in the sense that you’re dealing with a smaller source list. From that reason, Google News is more resistant. Any vertical or specialized search engine that deals with a subset of sites is SEO resistant (or more correctly, spam resistant).

Mahalo comes up next and how by using a small number of human editors, it can be harder to spam. Sure. So’s the Yahoo Directory. You remember the Yahoo Directory, right? It used, um, a small number of human editors to categorize the web. Advances in crawler-based search engines meant you could get really good relevancy and be spam resistant, which caused the Yahoo Directory to effectively be abandoned by Yahoo. Mahalo’s approach to custom-tailor the most popular searches is interesting — but despite heaps and heaps of publicity the new service has had showered upon it, it still hasn’t gained any real traction among searchers. Mahalo Launches With Human-Crafted Search Results from me in May describes the service in more depth.

Finally, Robert turns to Facebook, talking about the “social graph” term that’s now being bandied about as this month’s new Kool-Aid to drink. I’m being harsh — there’s obvious value in being able to look at the connections between people and form a ranking mechanism that can be applied to things. SocialRank, PeopleRank — whatever you want to call it. The idea at the moment is that Facebook especially has it, so everyone else better look out.

That leads to Part 2, where how adding the social graph to existing search technologies will really change the search game. Wow. Wow! I mean, yeah, never heard that before.

Look, Robert, back in January 2004, Eurekster launched with the promise of mixing the social graph with other search criteria, to improve results. As I wrote back then:

Personalized search? The concept has been that by knowing some things about you, a search engine might refine your results to make them more relevant. A teenager searching for music might get different matches than a senior citizen. A man looking for flowers might see different listings than a woman.

Eurekster’s twist on this concept is to provide personalized results based not on who you are but who you know. Friends, colleagues and anyone in your Eurekster social network will influence the type of results you see….

The potential of using your friends or colleagues is enormous. Imagine Eurekster being used by all the employees of a medical research firm, where many might do similar medical-related queries. With Eurekster, all the employees can be linked together and benefit from the searches and selections made by their colleagues.

Libraries are another institution that might latch on to the Eurekster concept. Librarians are constantly asked by patrons for assistance. Eurekster would allow librarians to collaborate invisibly with each other and share what they’ve found to be the best for various queries.

There are downsides. Not all of my friends have the same interests as me. In addition, as my social network grows — because my friends invite their friends and so on — commonalities that are useful get diluted.

Eurekster is still out there, but the idea of a network of friends influencing search results seems to have died at some point over the years. Viva not the revolution Robert was promising us.

Well, maybe Eurekster had no luck with that particular model since the company was small. Well, Yahoo’s not small. And in June 2005, it rolled out Yahoo My Web 2.0, which promised to bring social networks into search. As I wrote then:

After seeing what was planned, I remarked to Yahoo senior vice president of search Jeff Weiner sitting next to me that they were building “an eBay for knowledge.” Jeff was already literally bouncing at times with excitement in showing the new system, and the remark made him smile even more broadly.

He smiled because that’s exactly the Yahoo goal. My Web is Yahoo’s community rating system for information. Just as you buy things on eBay depending on ratings to know if you’ll trust a seller, My Web is what Yahoo hopes will help you choose more wisely the information you receive, whether you actively check reviews, contribute or remain an ordinary searcher who completely ignores the tagging and social search components.

In short, Yahoo’s not banking on tagging — the categorization of material — as a way to help people find things better. It’s banking that the mere act of saving things at all, even without tags, will give them a clue about what are trusted pages across the web. By looking at patterns of saving, Yahoo will have trust networks to tap into….

We’ve had a generation of search engines that depended on on-the-page factors such as word location and frequency. We’ve had a current second generation that tapped into link analysis, looking at how people are linking and what they say in links.

Personal search is that third generational jump, and Yahoo’s flavor of personal search is a social network one that it hopes will improve relevancy in web wide results in the way that link analysis helped drive back spam and improve relevancy years ago.

“We’re creating personal anchor text for pages, but by having a trust network, we can actually pretty much eliminate spamming,” Walther said.

Guess what? Still no revolution. The masses didn’t descend upon Yahoo My Web to form networks and save search results. In fact, Yahoo pretty much pulled back from the product, even dropping inline integration with it from regular search results back in October 2006.

But before Robert gets into applying social networks to search, he prattles on about Mahalo again being so superior to Google. Oddly for a video, he doesn’t show us any of the search results pages he’s talking about, saying at one point he can’t show us them. Apparently the camera he uses can’t be swiveled to show a screen. C’mon, Robert — if we’re investing the time (over a half-hour) to watch your video, make use of the medium.

He talks about a search for HDTV on Google versus Mahalo and pokes at the Google results, despite Google showing Wikipedia, How Stuff Works, CNET as well as a page from Amazon, just as Mahalo does.

I mean seriously, you might want to ding Google for deciding Amazon deserves to have a top listing for HDTV over all the other types of information that could be shown (especially if you’re one of the millions who use Google from outside the United States). But if you ding it for that, what’s up with Mahalo’s supposedly great human method of also deciding Amazon deserves a top spot.

Mahalo lists more than a top seven list, of course — you get a chunk of review sites, manufacturers, retailers and so on. But here’s the deal on Mahalo — it’s not really a search engine. The page it provides is good human crafted content, a good destination page like you might find at some of the other destination pages that Google lists. Mahalo — as Jason Calacanis himself will tell you — is a great place to start searching if your searches involve very popular queries. But if you want to hit those “search tail” terms that people always encounter? It’s not going to help.

My desktop computer crashed just when I got back from a trip, and since it’s likely to be down for a day or two, I decided to start using a new Vista laptop I purchased until I can buy a Mac replacement (heh — well, maybe not). But ZoneAlarm for Vista doesn’t block http referrer information as it does on Windows XP. That led me to do a tail search — block referrer plugin for firefox — something that’s only going to happen a few times per month, relatively speaking. Try it on Google, then on Mahalo. Plenty of good solutions right at the top on Google. On Mahalo, I have to wade through four “related” links that aren’t relevant (Ryan Block, Sunscreen, Netscape and Jet), then I get Google results.

Robert also tells us that Mahalo rocks because you know, the first thing you do in the search process for HDTVs is want to know the manufacturers. Bull.

First of all, no one can predict how someone else will go through a search process, so that’s bull strike one. But if I want to play magic mindreader like Robert, I’ll say the first thing people want are some guides to HDTVs. What is an HDTV? Is 720p enough to make a TV HD quality? Does it have to have HDMI?

Saying you first go to a manufacturer site in the process is like saying that if you want to buy a new car, just go visit some car dealerships. Me — I go to Consumer Reports, figure out the cars I might want from a third party trusted resource, understand the jargon I might encounter, and then I go to the dealership. And as someone who bought an HDTV last year, I also remember going to the horrible manufacturer web sites where they often provided only sparse info about their own products and certainly didn’t compare them to competing products.

We then learn from Robert that Google can’t change to be like Mahalo because it has algorithms that are “stuck in sand, stuck in cement” and shifting will “prove impossible.”

Insane. Seriously, like you want to scream stop talking. Robert’s a personally likable guy, but watching him make statements like this is like watching someone driving a car full speed toward a concrete wall while yelling “It’ll be OK — we’ll get through.”

Google has constantly changed its ranking algorithm over the years and will continue doing so. If Robert knew any SEOs, they’d tell him this firsthand. But more to the point, Google can’t change the ranking algorithm to be more like Mahalo because Mahalo isn’t using an algorithm to rank web pages — it’s using human editors. Maybe Google someday might get an algorithm to mimic much of what Mahalo does, but that still wouldn’t be the same as using actual humans.

So it is impossible for Google to change! Maybe the algorithm, but Google could easily hire editors of their own, pay them more than Jason does and do what Mahalo is doing if that model takes off — which, so far, hasn’t happened.

Robert then jumps into the idea that Google also can’t integrate social networking into its algorithms, pointing to Google’s largely failed Orkut social networking site as an example. He completely overlooks the fact that Google is playing the human/social aspect on a different level — personalized search, where results are refined based on what you as an individual seem to like. That’s a major shift for Google, and it’s also one that I’ve found personally compelling. For more on the service, see:

• Google Ramps Up Personalized Search
• Just Behave: Google’s Marissa Mayer on Personalized Search
• Google Search History Expands, Becomes Web History

In particular, Google has been talking about how personalized search allows for creating personalized PageRank (and see here for a patent look), a way where rankings revolve around what you personally like. It’s not a hard leap to extend that into a “social network PageRank” model, where if you define a social network, the collective interests of that network could be used to model the rankings. Google’s not doing that now, but to suggest that the mechanism are somehow impossible from either a company attitude or technological model is simply being ignorant of Google.

Finally — halfway into part two of his video, 23 minutes of covering all that’s “wrong” with some existing players, Robert unveils how social might be blended into Facebook, giving you the impression this is simply a “please hire me” pitch to Facebook itself.

First step from Robert, use old-style on-the-page ranking. Yeah, there’s a waste of time.

Here’s a thought — why not just license an existing search engine period? I mean, how do you search the web when on the Facebook site itself? You don’t. The Facebook search box only searches within Facebook (and despite claims from Facebook itself that it is some type of fantastic people search engine, I’ve found the search less than compelling).

Facebook is partnered with Microsoft, so it’s somewhat amazing (if not telling) that there’s no ability to search using Microsoft’s Live Search. Hit MySpace, in contrast, and you’ll see how the Google partnership has Google web search over there.

Facebook doesn’t need to build on-the-page ranking from scratch, not to mention the nightmare situation of trying to crawl billions of pages.

Next, Robert gets into the social network and trust aspect. Sure — an exactly like what Eurekster and Yahoo already promised. There’s nothing new here. Well, there is. As Facebook has grown, we’ve also had frustration grow — including the famed Facebook Bankruptcy that Jason Calacanis declared last month.

People have friends on Facebook who aren’t friends at all. It’s just easier to accept them. Robert, at the time of this writing, has 4,875 “friends” in the system. Really — he knows all of these people? And wants all of them influencing his searches?

Ah — but see, Facebook knows how to “lock out” the SEOs, Robert tells us, so he’s not overwhelmed by noise. Sure — but on the flipside, Robert is one of the top FBOs out there, Facebook Optimizers, to the degree people have been complaining about how his activities dominate the news updates that Facebook sends out to those with him as a friend.

There will be more FBOs, no doubt about that. Any system that has lots of traffic will attract people who will study ways to tap into that traffic. That’s good and bad. It’s good in that since it’s going to happen, you want people to learn appropriate ways to do this. It’s bad in that there will no doubt be spamming that comes along with it.

For such hype about his video, I was pretty much left with a “is that it” response? Facebook will get pages, then look at a social network and hopefully get those people to proactively rank pages when they search. Despite the fact that the Yahoo My Web experience tells me people don’t want to build search results — they just want to search.

Social network data applied to search does have promise. But to assume that social networks can’t be spammed and lack noise is foolish. To assume that people want to participate in actively shaping results is also mistaken, in my view. To also assume that major players like Google or Yahoo can’t tap into things that make Mahalo, Techmeme or Facebook good is shortsighted. Yahoo Answers is akin to Mahalo. Google News is Techmeme across multiple subjects.

By the way, Robert, if you’re tired of the SEO “noise” you think screws up results, then do this. In a search for scoble, for well over a year now, you’ve crowded out variety in the results by not redirecting your scoble.weblogs.com address to your new home at scobleizer.com. That means you get both results 1 & 2 for your new place as well as results 3 & 4. You also have no content at scobelizer.wordpress.com, plus another version of your old place at radio.weblogs.com.

You have contacts with the Weblogs folks — getting a redirect should be easy for you. You can kill or block that Wordpress site that you no longer use. I assume you maintain these other sites simply so that when people search for you by name, you crowd out anyone else from ranking well, perhaps people who might disagree with you on topics. That’s an aspect of SEO — it’s a tactic used as part of public relations in SEO. It introduces the same “noise” into the results that you cheered about not being present in Mahalo. So clean it up or cut it out with the SEO slams. You’re doing SEO yourself.

What about that New York Times article I mentioned? When Bad News Follows You is the article, another amazing “SEO sucks” story. The New York Times has opened its archives for crawling, which apparently is causing people to come forward at the not so astounding rate of one person per day complaining about articles casting them in a bad light. Blame SEO:

Technically complex, search engine optimization pushes Times content to or near the top of search results, regardless of its importance or accuracy.

Wow, seriously — did I just read that in the New York Times? SEO just shoves whatever crud it wants to the top of Google. Hey, think there are some SEOs out there that would like to rank for “new york times.” I guess they just need to SEO up some pages and they get there. Not.

Geez. The rest of the article does some hand-wringing on what to do to make the “right” articles appear at the top of the results (why not just sprinkle some SEO fairy dust on them?).

Insane. If an article is factually incorrect, then correct it. If the article is about someone with a negative connotation, then a later article comes out updating the story, link prominently from the top of the negative article to the latest version of a story. It’s called online journalism in the 2000s.

Postscript: I purposely haven’t read any of the other commentary on Robert’s post until I could brain dump my own thoughts. Since then, I read Rand Fishkin’s I Used to Respect Robert Scoble’s Opinion post, and he does a great job of poking back at the SEO attack as well as debunking Robert’s ideas on that Maholo-Google search shoot-out I mentioned.

Google and search from Dave Winer points out that spam is not the problem that both Jason and Robert like to paint it as. There’s spam, but there are lots and lots of great results, too.

Why Google Should be Scared of Facebook from Dare Obasanjo highlights that Facebook’s wall around its content is a threat to Google, but that’s a wall I think will get ripped down sooner rather than later, if only when Facebook decides it needs to show more ads on those pages and so needs more traffic.

Techmeme has much more commentary.

The Search Evolution So Far

Before going ahead, let me summarize what I covered in my past article, in terms of how search engines have changed over time to create and rank the results you get when doing a search:

• Search 1.0 (1996): Pages ranked using “on-the-page” criteria
• Search 2.0 (1998): Pages ranked using “off-the-page” criteria
• Search 3.0 (2007): Vertical search results blended into regular search results

The evolution above is not perfect. For one thing, some “Search 3.0″ blending started to happen years before 2007. It’s just that in 2007, I felt all the major search engines made the leap into Search 3.0 in a significant way.

As for Search 2.0, looking at off-the-page criteria such as links, Google kickstarted that heavily in 1998. However, some link analysis happened before then, and all the major search engines probably didn’t get on board to using it more fully until 1999-2001. But the launch of Google in 1998 remains the benchmark year in my mind, for that particular change.

The evolution is also only applicable to crawler-based search engines, those that use automation to gather web pages, store copies of them and search through the compiled index to create listings for searches. Yahoo was a major player using human power before 1996 and continued this way for years. Indeed in 1999, a majority of major search engines were presenting human-powered results. This quickly changed as Google grew. Yahoo made its human results “secondary” to crawler-based ones (then provided by Google) in October 2002. Today, all the major US-based search engines depend on crawler-based results.

To cap off the caveats, the evolution above is not the only way search engines can evolve. That’s just how things have largely gone with US-based search engines, which in turn tend to also be the major search engines for most countries around the world. There are exceptions. For example, Naver is the dominant search engine in Korea — and there, listings are largely human generated.

Search 4.0: The Human Factor

Onward to Search 4.0! As I said in my opening, to me this is the move for search engines to make use of human data as part of their ranking systems. In particular, it means human data generated by you, by those you know or by human editors.

Search engines already make use of some human data. All the major search engines, for example, monitor what we click on within the search results. This helps them determine if a particular listing is drawing more or less clicks than would be expected for the position it holds. For example, if the number two listing for a particular query is getting less clicks than “normal” for a listing in that spot, perhaps it’s a bad quality listing that should be replaced with another.

Another example: all the major search engines make heavy use of link data — and that link data is largely human data, humans both “voting” with their links and “tagging” pages by the words they use in the links. Google Now Reporting Anchor Text Phrases and Google Kills Bush’s Miserable Failure Search & Other Google Bombs provide more about how links are used in this fashion.

When I talk about putting human data into search results as part of Search 4.0, I mean things that are more aggressive or active than what I’ve covered above. I’ll start off with the most refined Search 4.0 implementation out there, Google’s personalized results.

Google: Search 4.0 Gets Personal

With Google Personalized Search, the web pages you visit, bookmark and things you click on within search results at Google are used to custom-tailor search results for you. The personalization is not as dramatic as with a place like Amazon, where if you purchase a book once, Amazon seems to continually push similar books like that at you forever. Shifts are far more subtle, mainly to help elevate results from sites you frequently visit.

To understand more, these articles go into depth about the process:

• Google Ramps Up Personalized Search
• Google Search History Expands, Becomes Web History

I’m fairly bullish on personalized search as an important addition to other factors (Search 1.0-3.0) in improving results. For one thing — better or worse — people often judge the relevancy of search results based on ego searches. Does a search engine find your home page, blog and related material when you search for yourself? Does it find your company? Personalized search is an ego search reinforcer. Because you go to your own places on the web often, Google senses that you want them to show up higher in search results, and they do. It’s a genius way to ensure anyone reviewing the service comes away pleased!

Of course, fulfilling ego searches can also be an relevancy advancement, not just a marketing ploy. There’s an excellent chance you’d have better searches if sites you visit more often get a bump in the search results. Personalized search can do this. In addition, over time, personalized search can potentially figure out other sites that are similar to those you visit and give them a relevancy boost.

Since Google expanded personalized search last year, there’s been one further major development. Personalized search uses searches over time to refine results. However, Google also has a system it is testing to refine results based on the last query you did, even if you aren’t taking part in the personalized search program.

“Previous Query” Refinement Coming To Hit Google Results explains more about how this works. It’s been used to improve the ads shown on Google for almost a year now, and it’s currently being tested to refine regular results. Google said that previous query refinement has been one of the strongest signals on how to personalized results so far.

Social Search: Promise Or Hype?

Last year, blogger Robert Scoble kicked off a round of “Facebook’s gonna kill Google” with a series of videos suggesting that because Facebook knows who your friends are, they’ll be able to apply that “social graph” data to improving search results.

The Promise & Reality Of Mixing The Social Graph With Search Engines was my response, a bucket of cold water explaining that using social data wasn’t some new idea that had never been tried before. The article went into depth explaining how Eurekster and Yahoo both assumed search could be “socialized” similar to photo sharing or bookmarking, only to find that wasn’t the case.

Yahoo had little take-up of its social search product. I’ve never seen the company explain why. My own suspicion is that take-up was low because search is NOT a social activity. I believe people tend to search when they have an immediate desire that needs fulfilling, and taking time away from the search activity to “share” with others is a distraction. Consider the person who has a broken water pipe. They might search quickly to find a plumber. They aren’t likely thinking at that moment that they want to tag and classify the search they conducted, much less the plumber they called. They just want the pipe fixed!

Eurekster has said that it found social activity worked better when people organized to build what it calls “Swickis,” search engines that hit only a custom collection of web sites related to a particular topic. Earlier this year, Eurekster formally came out of beta. However, the service has been entirely off-line for almost a week now. Practically no one has noticed, which speaks volumes to its usage and that aspect of the social search potential. Twitter, which some still view as a niche service, can hiccup for an hour and produce reams of blog attention. Eurekster goes silent, and the web stays silent about it.

I’m A Facebookholic & I Have 5,000 Friends

Still, couldn’t Facebook have more luck? For the record, when I spoke with Facebook director of engineering Aditya Agarwal about social search ideas last December, he was far more realistic than outsiders who hype what Facebook could do. In particular, he wasn’t certain how useful the social data actually would be for refining web search.

I plan to do a future article with Agarwal to explore this more. As a reminder, Facebook right now has no web search feature at all. And while it does have an ad deal with Microsoft, our previous Microsoft’s Facebook Ad Deal Doesn’t Include Search article covers how a search partner hasn’t been selected.

Let’s assume that Facebook does select a search partner, which it will need, since trying to index billions of pages and serve millions of queries each day is not an easy task (just ask Microsoft what it’s like to build that from scratch). What could it do with social data?

For one thing, it could monitor what people are clicking on in a potentially more “trusted” environment. Anyone can use web search anonymously, even sending in clickbots to make it seem like some particular listing is super hot. Having to register to be in Facebook and search from within there might make the clickstream data less noisy. But then again, it’s still a fairly open door that someone can walk through, if they want.

Facebook could tailor results based on what friends are searching on. If it knows what you and your 25 friends all seem to select from results, it could ensure those sites get ranking boosts for future searches. That’s very similar to personalized search, except it sounds full of extra friend-goodness, right?

The flaw here is plenty of people have friends on Facebook they don’t know. Some people collect friends for fun (and profit). Some people get friended by others just looking to build up their profiles. Some people you might friend not because you like them but because it’s easier to friend them than say no. Any of these instances can cause “pollution” of the social data that supposedly was going to improve your search results.

Consider also the case of someone who might work at some very conservative company but outside of work is a freeliving, devil-take-all person. Do they want coworkers who are friends to flavor their search results or those friends they hang out with when work is over?

Finally, privacy is an overlooked issue when it comes to social search. People often search for intensely private, personal things using search engines. Search engines are almost like confessionals, where people seek solutions to problems they might not tell real people that are close to them. With social search, do they have to remember to turn off a sharing feature that might be activated by default? And if it’s not on by default, will it get any take-up at all?

In the end, I think there is some potential to tapping into a social network and applying it to search. However, I still remains uncertain how that will unfold. It especially remains uncertain that this is somehow the secret sauce for anyone to jump past the current state of search.

Return To Humans: Hello Mahalo!

Earlier, I’d mentioned how Yahoo started off using human beings to create its search listings in the days before Google existed. Over time, the human soul in search was lost to reliance on the supposed scalability of machines. Anyone who wants to see how much we’ve handed over to machines need only search for buy cialis online on Google. At the moment, the results are littered with online discussion forums that have been “borrowed” by affiliates and others hawking deals.

Those pages will sit there for a day or two or three or potentially weeks, as Google usually tries to find an algorithmic solution to getting rid of them. The idea is you might have to suffer a bit in the short term until a long-term cure is found. But then like a virus that mutates, something else gets through, requiring a new long-term cure.

Enter humans. A human editor, reviewing results like that, can immediately spot junk that should get yanked. Even better, a human editor could act as a curator. How hard can it be to find 10 quality sites that should come up for that or other terms?

That exact human solution, of course, is what Mahalo has been banking on. Mahalo, launched last year, uses human editors to hand-pick top results. For background on the service, check out these past articles:

• Mahalo Launches With Human-Crafted Search Results
• Mahalo Greenhouse: Get Paid For Writing Search Results
• Mahalo Follow: Toolbar Gives You Human-Powered Alternatives To Searching, Surfing
• Mahalo Adds The Social Graph To Search
• Mahalo Adds More Social Features

As part of a talk I do on Search 3.0 and Search 4.0, I have some screenshots from last year that illustrate well how a human can indeed do better than the machines, for some queries. Remember the fires in Southern California at the end of last year. After a series of wide ranging ones, Malibu was hit with a second one a month later. Here’s what those searching on Google got in response:

The news box at the top is great, but sometimes searchers skip past things like this and go to the first “real” result. That’s a story about the Malibu fire early in 2007, not at the end of the year. Other results were largely about the fire of October 2007, rather than November 2007 (which is what many searchers at the time I snapped this would have been interested in).

Here’s Yahoo:

Again, news results at the top, then unlike Google, places you’d expect to find news about the fire — the local paper; ironically a map of the fires on Google Maps that Google itself didn’t return; the Malibu city web site, as well as the fire department.

Here’s Microsoft Live Search:

Very similar to Yahoo — a news box, the fire department, the Red Cross. What’s not to like? Well, let’s look at Mahalo:

Note at the top that Mahalo’s human editors understand there’s a different fire that happened in the past, in October 2007, and offer a link to a page about that. Then there’s a nice list of news sources, followed by coverage by date. Over to the side, a synopsis of the current situation. If you could see more of the page, there was lots of other categorized information.

It’s nicely done. It’s very helpful. And it was created with a human thinking about what other humans might want to see, rather than machines guessing.

Scaling Humans

So is Mahalo founder Jason Calacanis onto the Google-killer, human crafted results? No. I think human review can be part of the solution, part of the Search 4.0 addition to what we have out there already — but humans can’t craft pages for every possible search. In addition, it’s hard to keep those pages maintained once they’ve been made. It’s also easy to cross over from being a search resource that points to other resources to becoming instead a destination site. I think a good search engine avoids that (and Who’s Ranking For Knol? Hello, Wikipedia! has more on this topic).

Mahalo can also be overwhelming. Try a search for hillary clinton and there’s category after category. Background links. News links. Photos. Videos. Bio links. Blogs and message boards. Plus, there’s even more. I think at some point, you want your search engine to make some key choices for you, not flood you with so many that you don’t know where to begin.

Another issue is that what Mahalo’s human editors do, machines can get close to. Hakia especially stands out here. Search for hillary clinton there, and you’ll see how listings are grouped into categories like Awards and Biography without humans being involved (and see Social Networking Through Search: Hakia Helps You Meet Others for background on how Hakia works).

More Humans

There is another major search project involving humans: Search Wikia. Backed by Wikipedia cofounder Jimmy Wales, the service aims to involve humans in rating pages, annotating them and helping determine the ranking algorithm for choices the machine side of the project makes.

The articles below have more background on the service:

• Search Wikia: Not Even A Remote Threat To Google
• Search Wikia Adds Alpha 0.2 Features & More

Right now, the quality of the service is poor, as Search Wikia itself readily admits. There’s still lots of work to be done — and even with it, it might never succeed. But allowing humans into the process is, in my view, a good thing.

Indeed, even Google understands this. Last year, Google started doing some education about how human “signals” are already incorporated into its algorithm (see Google’s Human Touch and Google & Human Quality Reviews: Old News Returns). Aside from this, last year it also started testing a way for people to annotate search results — add those they like, remove some, suggest other ones.

Watch Personalized Search

Overall, there’s a role for humans, a way for them to be in the search process to enhance results. Actually, there will be several ways for them to be involved. Exactly how remains to be seen, of course.

Of the things I’ve outlined — personalized search, social search, human editors — I think personalized search is the one that will emerge as the major part of Search 4.0. That’s not to discount other things being tried, and they’ll contribute in some ways. But to me, personalized search has the most potential for another big relevancy leap. We’ll see!

In essence, Microsoft has taken the affiliate model – where merchants pay channel partners for leads which turn into sales – and turned all of us into potential partners. If it sounds like a crass play to buy your search allegiance, well, it is. But Goto.com was crass too, and it turned into a multi-billion dollar market, the ultimate expression of which is Google. So before you judge it, it’s worth thinking about a bit more deeply.

There’s no doubt that with Cashback, Microsoft is attempting to disrupt the search marketplace. But there are only a few axes around which you can do that. One, you can disrupt the presentation of search. This is very hard to do, but it’s happened before, and will happen again. Secondly, you disrupt the business model of search. And third, you can disrupt how search is created (ie, the secret sauce of relevance). There are startups along every one of these axes of disruption. But with last week’s news, Microsoft is focusing on the second one (business model). Unless, that is, you read between the lines. That’s when we see the beginnings of disruption along lines one and three as well.

But to the specifics, and I’ll circle back to that idea at the end of this post.

First, Cashback is a well understood evolution of the same direct response model pioneered by Goto. Instead of paying for a click, or lead (CPC, CPL), advertisers in the Cashback program pay only when there’s an actual sale. This CPA (cost per action or acquisition) model is well trod (Google rolled out a limited version two years ago, and there are no shortage of companies playing in this space), but in this case the action is a sale, and this is the first time a major player is providing incentive directly to the person who is doing the searching.

The reaction to Microsoft’s news this week was decidedly mixed. Obviously, the easy (and accurate) criticism is to point out that Microsoft is attempting to buy its way back into the search game. But according to the Microsoft executives I spoke to last week (some of which were off the record), this is about far more than buying back search share (Microsoft’s US share is around 8-9%, Google dominates at 65-68%). Instead, Cashback is an attempt to disrupt the markets by shifting the value proposition from one of harvesting consumer intent and delivering leads to advertisers (Google’s model) to one of harvesting commercial intent and delivering sales to advertisers.

This is where Microsoft’s purchase of Farecast comes in. Lost somewhat in the analysis so far, I think Farecast is a key part of Microsoft’s strategy – it’s a disruption along the first axis of search – how search is presented. Those of you who have read Searchblog for a while may recall my initial post on that site: Rip Me Off No More. It really struck a nerve, I had more comments on that post than nearly any other in the history of my site. Turns out, people really like a search engine that promises to 1. help them find the best price and 2. does it in a trustworthy, intelligent, and timely fashion. From my coverage:

What Farecast does is shift the power of information back into the consumer’s hands, and that’s why I like it. I remember when the web was young and the first car buying sites were up and running. Dealers scrambled for that early business, and I bought two cars off the web by forcing dealers in the Bay Area to compete for my business. It really felt like the web was going to change the dynamic of who was in charge in a car buying transaction – because I could force dealers to their best price, I was always going to get the best price. It felt like this would be the model in most large transactions, like travel, loans, etc. Price would stabilize, and folks would differentiate on service, relationship, and approach.

But something funny happened on our way to internet mediated bliss: the big companies figured out how to game our demand. Dealers realized they can make more profit if they cooperate and withhold pricing information from the aggregators, and the aggregators got into bed with the supply side of the equation (if you think AutoByTel or Expedia is on your side, you’re kidding yourself). Nowhere is this more true that in how an airline prices its tickets.

I like how Farecast puts the consumer back in control of the data. The interface is very slick and the idea is quite promising.

Promising enough for Microsoft to buy the company, in fact.

OK, so where is all this going? Well, let’s start with the full suite of Microsoft’s news last week. Sure, there’s Cashback, but that’s not all. Microsoft also announced that Farecast was, well, Live (ie, now a travel search vertical on Live search). This points to a longer term strategy: create (or buy) the best in class vertical search engines, then tie them into Cashback. So far, Farecast is not integrated, but I am sure it will be. And the Farecast engine can work for a lot more verticals than just airfare. It’s just data and software, after all. And that’s the business Microsoft has been in for a long, long time.

Secondly, let’s look at the list of partners who launched in support of Cashback: “Key partners participating in the Live Search cashback offering include Abe’s of Maine, B&H, Backcountry.com, Barnes & Noble.com, Circuit City, Cookware.com, Crutchfield, eBags, eBay, Foot Locker, GiftBaskets.com, The Home Depot, HP, Jockey, J&R, Newegg.com, OfficeMax, Overstock.com, PetSmart, QVC, Sears, Spiegel, TigerDirect.com, and Zappos.com.”

So which one jumps out at you? If you said eBay, you win. eBay is still one of Google’s largest customers, but thanks to Google’s Checkout program, it’s not a exactly happy one. eBay’s already hedging its bets in Europe with Yahoo, and it’s more thank willing to do the same with Microsoft via Cashback. Watch this space closely. If Cashback starts working for eBay, eBay will start spending less with Google. And you can be certain that Microsoft has every intention of making Cashback work for eBay. Not to mention, eBay has every reason to work with Microsoft to limit Google’s power.

So let’s circle back to the original idea. Is disrupting the business model by paying search customers when they buy something a good idea? I think it is. But it’s not going to work unless we trust the search results in the first place. That’s where Farecast comes in. In the short term, Cashback will probably goose Mircosoft’s user loyalty numbers, which should also boost its share of searches overall. But longer term, the key to winning will be the integration of Farecast-like innovations into Microsoft’s offerings. I’d look for these to come in the next year, if not sooner.

Lastly, if Microsoft really wanted to disrupt the market, it should look at turning Live search, including Farecast, into an open development platform (I see this coming from Yahoo already) – in short, disrupting the third axis of search – how search is created.

But that’s the subject of my next post. Stay tuned!

Search 1.0: Location, Frequency, & On-The-Page Ranking Criteria

Let’s go back to the good old days, around 1996, when some of the big names in web search were Lycos, Infoseek, Excite, WebCrawler, Open Text, Hotbot, and AltaVista. Yahoo was there, too — but it was the exception, the search engine that relied on humans to catalog the web, rather than the others that used “crawlers” to automatically build listings.

Those crawlers were robots that automatically followed links across the web. When they came to a new page, they would effectively make a copy of it. All the copies were stored in an “index,” which was like a big book of all the pages they had collected. The crawlers indexed millions of pages, which sounds like nothing compared to the billions of pages that are harvested today. But back then, the web was a lot smaller!

When someone did a search, the actual namesake “search engine” component kicked in, using an “algorithm” to sort through all the pages in the index. The algorithm was a system designed to return what a programmer thought the best pages would be. Back then, the best pages were deemed to be those that used the search words a lot in proportion to other words on the page, plus whether those words appeared in key areas of a web page.

Consider a search for shoes. The algorithm might not like this page:

Walking Alone At Night

Walking alone
I hear behind me
Footsteps
Shoes softly scuffle
Friend or foe?
Keys, and I’m safe inside

Maybe the algorithm doesn’t like bad poetry written in a hurry. But see how the word “shoes” appears only once? How relevant is this page really, then, if shoes only gets a single mention. Now consider:

Buying Shoes Online

As the web grows, many are discovering
that shopping can indeed be done online
for all types of things, even shoes!

Yes, footwear fanatics. You can buy
shoes online. But what about trying
them on? Those selling shoes have
thought of that. In this review, we
look at online shoe sellers with great
return policies.

In the second example, shoes is mentioned several times. Plus, it appears as the headline of the story, and in the opening paragraph. Let’s say it also appears in the HTML title tag. That means shoes not only appears with frequency, but it also appears in key locations on the page. Algorithms back then liked that.

So first generation search? Search 1.0 was largely about looking at the location and frequency of words on individual web pages to measure them against each other.

Search 2.0: Looking At Links & Other Off-The-Page Criteria

It didn’t take long for savvy search marketers to figure out that if the algorithm liked certain things, you should tailor content to suit. Thus, SEO was born — search engine optimization, optimizing content for search engines. Some SEOs focused on making relatively minor changes that still could have dramatic impacts. Other figured if location/frequency was what the search engines wanted, the more the better:

Shoes, Beautiful Shoes!

Shoes are the best shoes
anyone could hope to find
shoes online at our store
with shoes shoes shoes
blue shoes red shoes
walking shoes tennis shoes

Pretty ugly — but that’s OK — this text might be hidden from human view in a variety of ways. The main point is that location and frequency worked better in the trusted environments of libraries and controlled collections of documents that web search grew out. Applied to the open web, where anyone could get anything listed, it started to fall apart. Back in 1999, AltaVista founder Louis Monier told me:

The good answer to the query ‘car’ has nothing to do with the text. I’d rather use a medium with a crystal ball.

Instead of a crystal ball, two guys by the name of Larry Page and Sergey Brin had fired up their own search engine, Google, that relied on link analysis to rank pages. They weren’t the first to look at links, and PageRank was only part of the overall algorithm. But the pair popularized a shift for all the major search engines to make a second generation leap — depending on criteria not actually on web pages to rank which pages are best. Call it Search 2.0.

It’s not just links that can be part of off-the-page criteria. Clickthrough is a metric that has been and may still be used, where you determine if people are clicking on a particular listing more often than one in a particular position normally gets. If so, perhaps the listing should be ranking better. Domain age and general traffic levels to a site are other things that can be used. But it’s really the links that remain dominant.

Google in particular popularized the notion of links as votes, with the web being “democratic” in how those votes were cast. And for a time, it worked. But then we got active campaigns to win votes, Googlebombing that led to things like the miserable failure ranking for President George W. Bush. We also saw an economy of buying and selling votes spring up, which to this day sees Google trying to fight off paid links.

Search 2.0 still works, of course, despite these problems. But things could be better — cue Search 3.0 to the stage, please.

Intermission: What The Hell Is Vertical Search?

I’ve long hated the term “vertical” search, which was popularized by financial analysts more than search engines. I hate it because most people have no idea what it means. I know. I ask audiences all the time if they understand the term, and plenty of hands of those indicating “no” go up, followed by “ah ha” looks after I explain it.

But vertical search as a term is here to stay, continuing to be used and even having its own Wikipedia entry, so let me explain it.

Go to Google, Yahoo, Microsoft Live, Ask.com and do a default search. What you’ve done is a “horizontal” search. You’ve searched across the entire spectrum of the web for a topic. To illustrate, consider just some of the topics that pages are about:

… shopping video news sports weather research stocks lottery …

See, a nice long horizontal line. But if there’s some news event that happens, you don’t want to search the entire web for information. Doing that means you’re hunting through billions of pages to find the relatively few updated with the fresh and relevant information. Far better if you focus only on news content.

To help you do that, search engines build specialized search engines that only go to news sites. Rather than search across the horizontal spectrum of the web, they let you slice down “vertically” only into news — or into sports — or whatever. Search for “fires” in a news search engine after a major fire in a particular area, and pages about that fire will dominate the results, since you can apply a time-based ranking criteria. Search for “windows” in a home improvement search engine, and you shouldn’t get information about how to fix Windows Vista.

Vertical search isn’t new. However, it faces a huge challenge. Searchers simply don’t know vertical search engines exist. Put tabs, buttons, drop-downs, you name it — it’s been tried (see Why Search Sucks & You Won’t Fix It The Way You Think for some illustrations). Users have ignored these options.

Here’s another problem. Consider this:

That’s my “Google 2005″ image that I made in 2001, for a keynote to a librarians’ group on search trends. My slide was to illustrate that Google couldn’t keep adding tabs for each vertical search engine it unleashed. More important, even if it did, no one clicked on the tabs. Instead, I explained that search engine would need to learn to push the right tabs for us, behind the scenes, something I eventually called “invisible tabs.”

Search 3.0: Blending Vertical Search Results

Even in 2001, we had some of this mixing of vertical results happening, usually at the top of pages in what Google called OneBoxes and others gave different names. But finally this year, search’s third generation really happened. The supremacy of “horizontal” web search results being the default was hit by Google’s Universal Search rollout.

Our Google 2.0: Google Universal Search article from May explains more about the inner workings of Universal Search — and the headline might now seem odd — shouldn’t it be Google 3.0 if we’re talking Search 3.0? Well, Google 1.0 was a Search 2.0 search engine already! That’s my story, and I’m stickin’ to it.

The short story is that Universal Search replaces some of the web search results with listings that come from vertical sources, such as video, local, or news. The results are mixed in, blended as appropriate, as our Search Illustrated column, um, illustrated in July:

It’s not just Google, however. In June, Ask launched its Ask 3D interface that used the “Morph” algorithm to automatically decide which vertical search results to blend into the main page. Yes, web search listings still remain front and center, with vertical search blended around web search — but vertical results became far more prominent.

In September, Microsoft relaunched with Search 3.0 features and in October, Yahoo finally announced Search 3.0 material it had been rolling out.

OK, I am stretching the last two a bit. In both cases, they’ve mainly enlarged the “shortcut” style material that shows up at the top of a search for some queries, such as illustrated below for Yahoo:

And here for Live:

Still, the vertical search listings are growing in how often they appear at Microsoft and Yahoo, as well as with the amount of space they take up on the search page. It’s enough for me to declare Search 3.0 now fully engaged by all the major search engines.

For search marketers, Search 3.0 represents new opportunities. There’s less content in various vertical search engines, so the odds of ranking well naturally increase more. In addition, much of the content in vertical search areas such as local and video seems poorly optimized. With just a little care, people should be able to see improvements — and now improvements that may bring them to the first page of the “regular” search results.

Want to learn more from a search marketing perspective? Aside from how we cover verticals (and thus Search 3.0) through our columns here and features, as I said above, we’re going to have an entire track on Search 3.0 at our SMX West show happening next February, specifically these sessions:

• Search 3.0: The Blended Search Revolution
• Search 3.0: Video, Images & Blended Results
• Search 3.0: Local Search & Blended Results
• Search 3.0: Online Retail & Blended Results

Search 4.0 & Beyond

I’m certainly not the first to use the phrase “Search 3.0,” and I know not everyone agree with how I suggest it be used now. Time will tell! But I hope this article explains why I find it useful to explain the latest generation that’s been unleashed, especially in a world where everything seems more easily understood by some if it’s X.0 something.

I’d note that back in October 2005, Robert Scoble wished for a “Search 3.0″ world where search engines learned from users, and Read/Write Web wrote in July about “Search 2.0″ companies that were using “third generation” social features. That got me to comment:

The third gen has commonly been considered the combination of personal data — either refining results because of your own past history or that of others. Lump it all into social search, and that’s your third gen. And since it’s third gen, please call it Search 3.0 if you must and don’t force it into a Web 2.0 world just to try and mesh some Web 2.0 companies into a Search 2.0 framework.

But the human / social element does have a role to play in search. It’s the other long expected evolution to hit, one that I’ve long written about being alongside vertical search as a third generational leap:

Vertical search is important because it’s one of the two major things I’ve long talked about as being how search will advance. First generation search analyzed words on a page to rank content. Second generation search tapped into link analysis. Third generation search to me is looking at both user input (what we visit; what we click on; personalized results) and making search go more vertical.

So where’s the humanity in Search 3.0? It’s not — I’m shoving it into Search 4.0, a fourth generation leap that Google’s already making and the other major search engines have yet to push into.

Search 4.0 is another track at the show — and the topic for another article next week. Next time, more on personalized and social search. Until then, here’s some key reading for those who want to look ahead:

• Google Search History Expands, Becomes Web History
• Mahalo Launches With Human-Crafted Search Results
• The Promise & Reality Of Mixing The Social Graph With Search Engines

In the past few years, a significant new feature has crept into the results portion of this otherwise predictable interface. Called “universal search,” the idea is to incorporate more than simple HTML pages into the results. A search for “London restaurants”, for example, might bring up maps and local results, as well as videos, images, organized reviews, and of course web pages. Every major search engine, from Google to Ask, has incorporated some kind of universality into its search results.

But while universal search points the way toward a new approach to getting you the answers you seek, it’s a half step at best. The results change, somewhat, but the process is pretty much the same. You enter a query, you get a set of results. Not particularly new.

What I find interesting are entirely new approaches to the interface of search. What happens when search is no longer driven by the command line and the blinking cursor? What happens when, for example, your query is informed by where you happen to be, or who you happen to be, or what you happen to be doing at the time of the request? To explore these ideas, it’s best to step outside the current box of a web browser on your PC, and think about mobility.

To help folks think outside the PC box, I often use the example of buying wine at a supermarket. Imagine you’re in front of the wine aisle, staring at all those expensive bottles. You choose one at about eye level, knowing that the higher a bottle is, the more expensive it becomes. Are you getting ripped off, you wonder?

This is where search gets interesting. Instead of noting the vintage and vineyard, driving home, and Googling the wine, you whip out your phone. You point your camera at the label, click, and – voila – that’s a search. You turn your screen back to your gaze and there are your results: “2001 Stag’s Leap Cabernet. Avg price: $48. Avg. rating (friends): 8 of 10. Avg rating (Wine Spectator): 90 of 100.”

You turn the bottle over in your hand, toward the back, where the price tag is. $76. Yep, you’re getting ripped off.

But that’s not all. The results include these links: “Click here to find this product near you. Click here to buy. Click here for delivery options. Click here for detailed history of this product’s ecological impact.”

You click on “find the product near you” and see that it’s available, for $45, at a nearby liquor store. And for a few bucks, they’ll even deliver it to you. A few more clicks, and the bottle is on its way to your home. Satisfied, you place that $76 bottle back on the shelf and saunter out of the store.

Now that’s search!

Is it possible? It most certainly is. And it’s not far off. But the point of this story isn’t to show you how you might save a few bucks – it’s to demonstrate how flexible and powerful search can become when it is unhinged from the standard approach to which we’ve all become accustomed. When it comes to search, change is most definitely good.

Blekko

What we know so far about Blekko isn’t much, and TechCrunch has the most details in its The Next Google Search Challenger: Blekko post from yesterday. Apparently Rich founded the company in September 2006, along with five other former Topix employees, after he left Topix in June.

Rich told TechCrunch not to likely expect anything public until 2009. I agree with Michael Arrington at TechCrunch that Rich has a track record that makes him well worth watching. The Open Directory was an initial success, though the model didn’t scale well. Some of that was within the founders’ control but had more to do with AOL’s lack of backing. The company should be dragged into the International Court Of Search Crimes and be forced to sell the ODP to someone who will support it properly. Topix has built a reputation and is still standing and succeeding — though I’d say it still has far to go to seriously threaten Google or Yahoo.

Rich adds a bit more in his Why Search? post today:

Having just spent 5 years in the media space, I’ve come away with the idea that editorial differentiation is possible. But the editorial voice of a search engine is in the index…so it has to be algorithmic editorial differentiation.

So far, it doesn’t sound like a social networking play like some of the others. We’ll be watching, Rich. Also see discussion today on Techmeme.

Powerset

Powerset is now a classic example of why you WANT to be a stealth start-up and say little. That’s because when you get too much early press — in part through your own doing — then fail to deliver anything, the hype can swing back at you hard.

The company came to light back in October 2006 via VentureBeat, with the twist being that natural language search would be the way forward. That caused me to write a long rant about the hype of natural language search in reaction. From the top of that:

This is a rant. It’s a rant from over 10 years of watching people trot out natural language search as the “killer” solution to the current state of search, something that’s happening once again with Powerset. That’s a search engine you can’t even use at the moment, but the hype will no doubt continue. To counteract that, my thoughts on and some history about natural language search.

Natural language search makes a compelling pitch for those who really don’t know search or haven’t heard the natural language mantra before. I’ve seen the pitch time and time again. You:

• Pick out an example that shows how “bad” search is on an existing search engine
• Demonstrate how natural language search would work better on your service
• Sit back and collect the press attention

I then went on to detail how natural language search had been hyped and tried over the years. The short story is this: It doesn’t take much natural language analysis to figure out what someone wants when they type in “britney spears nude” or “hotmail.” In addition, by and large I don’t believe enough people will change their basic search habits to enter long sentences when searching any time soon.

Since that time, we’ve pretty much had nothing out of Powerset other than the launch of Powerset Labs in September 2007. That launch hasn’t produced any cool applications that I’ve seen or heard about, nor much buzz. Instead, in November, we got a management shake-up.

For a more formal chronicle of the company’s developments, check out this area at VentureBeat and these search results at TechCrunch.

Finally, while I’m harsh above on Powerset, I actually had a long visit with the company in the middle of last year and was deeply impressed with the effort going on there. I’m still working on a long write-up to explain what’s happening. But in a nutshell, Powerset is trying to literally comprehend or understand each page on the web.

Today’s search engines don’t know what a page is about by reading words. They’re more or less doing pattern matching — finding pages that contain words similar to what you search for (or pages relevant to those words based on linkage). Powerset literally is trying to read and understand what a page is about the way a human reads a page and knows it is on various subjects.

I don’t see that as making it a better search engine that Google. Instead, I think it may eventually give it the ability to create a unique “auto-Wikipedia” style site, assembling knowledge pages on any subject automatically. I also think that there will eventually be some search benefit in comprehension of pages, but exactly how that will play out I suspect is part of being with an existing search engine and a more traditional model. With the array of patents Powerset has lined up, I suspect it will eventually get acquired by Google, Yahoo, or Microsoft rather than rollout its own product. But we’ll see.

Hakia

Like Powerset, Hakia has played the natural language search game. Unlike Powerset, it has a product anyone can use — live since at least the middle of 2006.

Again, I’ve been working on a long write-up on the inner workings of Hakia and have yet to finish it. It’s complicated, and I mainly want to cover what I find to be the real use of their technology — the ability to create custom “gallery” pages and understand those are related to particular searches.

It’s easier to show you what’s impressive. Search for hillary clinton, and you get a nice page showing news, her official site, biography pages, blogs & fan sites, news & interviews, and more. It’s very Mahalo-like, except it doesn’t require human editors like Mahalo and predates Mahalo by a year.

That categorization is something I know the major search engines could do, if they wanted. So far, they don’t. And so far, despite Hakia talking about its rising traffic, it has yet to make a serious mark. Moreover, in October, it made a serious shift to allow social interaction with its results. That’s a sign that the original plan that “natural language will win all” has failed to do so; therefore, another twist is needed.

Social Networking Through Search: Hakia Helps You Meet Others from Vanessa Fox here at Search Engine Land covers the change, plus it gets into the natural language indexing stuff I mentioned earlier that makes Hakia unique, plus has examples of gallery pages.

Mahalo

Credit to Jason Calacanis. He said he wanted to take on Google, then wasted no time getting Mahalo rolled out. OK, he also says he’s not taking on Google — just focusing on the top searches that he thinks would be better with human review. Sure, you aren’t taking on Google, Jason.

To date, Jason reports that Mahalo’s traffic is growing and strong. But to date, I’ve certainly see no webmasters taking about what a traffic driver Mahalo is. It would be early to call it a raging success, but it’s a nice alternative to have. Indeed, later this month I’ll finally finish my Search 4.0 piece that picks up from the conclusion of my Search 3.0: The Blended & Vertical Search Revolution article last November. I’ll show some examples of how the human element at Mahalo can and has kicked some Google and traditional search engine butt — though also how it isn’t the panacea some expect.