• http://www.mortgageclass.com MTing

    Arnaud,

    Very insightful post but I do have a comment regarding the direction of search…

    If social search means that the internet is going more democratic, do we not run the risk of establishing “herd mentality”? Where as with any group dynamic, there is always a small group or individual who begins to take over. Then the rest of the group simply follows along. Does that necessarily mean that “the wisdom of crowds” is a good thing?

    I am also not entirely sure how this movement is “slowly but surely leveling the playing field for the rest of us in the tail.”

  • http://searchistheos.blogspot.com/ Arnaud fischer

    MTing, right on! Glad you enjoyed the piece. James Surowiecki actually defines The Wisdom of Crowds around a few principles. The principle of opinion Independence is critical. There cannot be influence or as Surowieki explains, “cascading” effect from one to the next, so … no “herd mentality”, theoretically. We all can think of counter historical examples, of course.

    -arnaud
    Search is the OS!

  • http://www.cameronolthuis.com Cameron Olthuis

    Great article Arnaud!

    Re: Social search… Wouldn’t you say that the long tail is one the flaws of social search? If X number of queries are only searched 1-3 times then the crowd isn’t big enough for the wisdom on those results.

    In my opinion, Google(or Y & M) already has all the info they need for social search. We vote with our click-streams, time spent on pages, etc… G already has that info.

  • http://dmoz.org/profiles/chris2001.html chris2001

    “The principle of opinion Independence is critical. There cannot be influence or as Surowieki explains, “cascading” effect from one to the next, so … no “herd mentality”, theoretically.”

    Hm… then maybe we should stop trying to use the “wisdom of crowds” as base for earthly search applications – we´d end up with tools that need to be manned by angels, who have access to pure reason itself, ínstead of humans ;-)

    Typical characteristics of modern democracies, that are important for stabilizing them and preventing that herd instincts take over, are standards (e.g. human rights, constitutions and laws of specific states) and institutions (e.g. courts of law, schools that communicate standards to young citizens). Applying this to the development of “social search”…

    Standardization: Successful “social search” projects will usually meet growing challenges in form of poor quality contributions, abuse and spam, which enforce the creation of standards for their specific core task (e.g. tagging, digging, article- or review-writing). These standards, once established, can help a lot to keep herd behaviour under control.
    Beyond the limits of specific projects… the more colourful the social search scene becomes, the more important it will be that the community of net citizens and the various search market players develop shared standards. E.g. what are basic requirements for “quality content”? What are basic criteria for “spam”? Without a minimum agreement on questions like these, collecting user input, sharing data or creating mashups will get more and more difficult in the long run.

    Institutionalization: Specific initiatives (usually Open Content and Open Source projects) will be charged with tasks which are very important both for the community of net citizens as a whole and the search market, but at the same time require such tremendous human resources that the community can´t afford doing them three times. The best already existing examples are writing an encyclopedia and maintenance of a general web directory. If you have a closer look how Wikipedia and ODP are used today, they already play a role that reminds strongly of an institution: their data are used for science, to seed new applications, or as a correction factor or reference standard against which results of other projects and tools can be checked.

  • Mark

    “According to Jupiter, 41.2 percent of users report that general search results are often not directly relevant to queries, and 18 percent leave a search engine without having found the information they were seeking.”

    Could you please give more information on the source for that? Thanks!

  • http://www.theratingsguy.com RatingsGuy

    Social Search is a means for consumers to avoid advertising, SEO placements, and congestion within search. People trust other people more than they do advertisers of all types.
    Secondly, it is basic human psychology for consumers to educate first (using search) and validate second (getting opinions). Social search helps consumers move through validation and on to decision making. Anyone who helps facilitate the validation process to determine; trust, value, and quality will help and enhance consumer decisions. Consumers are not blind to G,Y,& M’s advertising keyword bias, and will seek alternatives in a NY minute for the second phase of decision making. IMO G,Y & M fail the consumer in favor of being a better Yellow Book.

  • 2valhalla

    Arnaud,

    Great read. Is it possible that users/contributors will pull value out in other ways than simply getting paid by posting video on YouTube or searching? Maybe the value actually resides in relationship (both tangible and intangible value). For example, if you are using a social networking site like http://www.lyro.com you might get intangible value from making a new business relationship. Or, if you get an RFI/RFQ/RFP, you might actually recognize tangible ROI.

  • http://searchistheos.blogspot.com Arnaud fischer

    Cameron, thank you for the comments and feedback; fully appreciated. You are very right regarding the tail’s query distribution. Many queries actually have a frequency of 1 or don’t even show up in search logs from one month to the next. There are over 10 billion docs inventoried by the major crawlers out there and only about 700 or 800 million unique users worldwide.

    Social search is complementary, additive and augments traditional algorithmic search. Social search is not a substitute to traditional search … just making search as we’ve known it potentially significantly better.

    -arnaud

  • nickd

    I agree that there appears to be a fundamental contradiction in the claim that “The wisdom of crowds – so well articulated by James Surowiecki – is at the root of emerging information retrieval tools”.

    In Surowiecki’s book, the successful examples of the use of crowd wisdom require independence between the members of the crowd. The math that underpins this is Bayesian Search Theory (c.f. The Theory of Optimal Search, and/or the classic story of finding the lost nuclear sub Scorpion). Surowiecki in fact offers lots of examples of where crowd wisdom doesn’t work – most all of which involve biases due to lack of diversity of opinion or independence of the crowd members.

    However, social networking, community based recommendation engines, blogs, voting/ranking, etc. all seem to violate the key enabling principle (independence) that allows one to apply Bayesian Search Theory (aka wisdom of the crowds) to the judgments of the participating individuals.