Sometime later today, a small number of Google users will see a new look to Google’s Search Options feature. If all goes well, the cleaner display may be launched across Google after the New Year. And it’s all because Google’s vice president of search product and user experience Marissa Mayer doesn’t like jazz.

Simmer down, jazz lovers! Jazz is just not her thing; she’s not making a personal campaign against it. Instead, Mayer was using jazz to explain a pet metaphor she has about search results pages. They have their own “rhythm,” and Google’s results have been sounding a bit free form lately.

“I don’t like jazz, because you never know what’s going to happen next,” Mayer said, continuing on to apply the musical style to Google’s search results. “I’ve been calling this problem ‘user interface jazz.’ This result looks this way, and that result looks that way [something much different], and it really does slow you down.”

Indeed, it has felt a bit confusing with Google lately. When I wrote my Up Close With Google Search Options story in October, after new search option features were introduced, I detailed a number of inconsistencies in how they operate.

I haven’t yet finished another piece about a growing gripe I have, inconsistency in how Google enhances search listings with sitelinks. I never know where to expect them now. They can appear in in any position, within snippets, on a single line and in even more ways. It’s been making my head hurt.

I raised the inconsistency issues with Google cofounder Sergey Brin last month at a press conference, and he said experiments on this were in the works. So now we have a visual sign of that.

Search Modes

Personally, I’m glad if the results will be getting more predictable. But enough generalities. Let’s take a closer look at the changes. Here’s an overview of the new results page in testing (and yes, you can use the screenshots below if writing about this — just link over to our story):

“We’re basically looking at a new look and feel for Google,” Mayer said of the change. “It’s an overall cleaning up of the search results page.”

The search options appear in the left-hand column. The former “All results” area that allowed you to switch between different types of searches (images, news, maps and so on) has been replaced with new tabs for these services:

Internally, Mayer said that Google calls “modes.” For example, after searching for “australia” in the example above, you’re in “Everything” mode (I love this name). But with a click on the Images tab, you can switch to “image mode” and get back image results, select News to get news results in “news mode” so on.

By default, Google guesses at the modes it thinks are most relevant to your search. But the “More” tab gives you access to the full range of search services Google offers. If you Video mode, and that’s not automatically suggested, you can select More, choose Video and get those results:

As happens now, when you switch modes, the search options change. In video mode, you get unique video filtering options such as duration of clip or to see only video in high quality:

New to the search options area in this test is a “See also” section that suggests other queries related to your original topic. As for a search on pizza, Google also suggests things like “tacos” or “fried chicken:”

Interestingly, it doesn’t show more typical related queries that incorporate the main terms, as you can currently see at the bottom of results now for pizza, such as “pizza recipe.” But perhaps this will change.

As the second arrow in the screenshot shows, there are even more search features that you can access by using the “Show search tools” option. Features such as “Wonder Wheel” or “Timeline View” or “More shopping sites” all reside in this area (to learn more about these, see Up Close With Google Search Options).

Google 3D

The most dramatic change in the design is that the search options window is permanently open, rather having to be manually toggled on, as is the case now. That’s right. If the test proves successful, Google’s almost certainly moving to a three “pane” format, with search tools and options located on the left, search results themselves in the middle and ads on the right.

Mayer said going left made the most sense. Google continues to add new search features, and they need to be exposed to searchers somewhere. Putting them at the top of the page pushes results down; ads are already at the right. Having the tools on the left, with the pane permanently opened, is something she said she’s personally wanted for some time, but not everyone in the design team was convinced. The test will be a final proof of how well it works with Google’s audience.

“While I wish we had gotten here sooner, Mayer said, “I’m excited to finally have it out.”

But hey, wasn’t there a search engine that ushered in a three pane design like this not to long ago? Yep. Ask.com, with it’s Ask 3D view that was developed under then-Ask CEO Jim Lanzone. Bing and Yahoo now have three pane designs, as well. So did Ask have it right back then, I asked? And is the three pane view now the industry standard?

“Now you know why I’ve been searching for John Stuart Mill,” Mayer said, speaking of his book On Liberty and how it discusses that universal truths always come through.

“Is it the trend du jour or is that a universal truth,” she said, of a three pane design. “It is a likely universal truth.”

Mayer added that Google had recently done nearly 50 mockups, all done independently, to examine results. A three pane design was a common theme that kept coming up.

Bimodal World Of Screen Sizes

Mayer also spoke to the growth of screen sizes, pointing out that we’re in a “bimodal” world where screens are conversely getting larger on the desktop (and people have more of them) and smaller (as people do more and more browsing on mobile devices). So a three pane view may make no sense for mobile devices. But on the desktop, there’s much more room to spread out than in the past.

“This is a first step. We do think we can add that left navigation and it won’t impact the rest of the page,” she said, pointing out that currently, the actual search results only take up about 1/4 of the screen real estate available for a typical desktop viewer. She also said that Google will do more things in the future to take advantage of larger desktop screens.

What about the navigation bar at the top of the results, which people can also use to switch between different search options such as news and images. Is it becoming redundant as the left-pane continues to evolve?

Mayer said Google would revisit the design of the top navigation links area next year. But right now, it works, moving around “a ton of traffic” to different portions of Google. She also said it’s a helpful way to unify Google’s various properties. Plus, it allows people to go directly to a particular search service, such as image search, without first having to do an “everything” search and then refine it.

Search Button, Meet Search Box

Looking at the new design, I remarked how noticeable it was that the search button is directly integrated into the search box. Consider the before:

And after:

Internally, plenty at Google have noticed the change and not necessarily liked it. “That’s has been one thing that’s drawn the most ire. If the ire continues [from the public testing], that may be one of the first thing that changes,” Mayer said.

She added that Google’s also closely looking at how well the Everything tab and the new Search button work, as illustrated below:

“That big blue Everything on the left and search button on the right, they do pull …. and we do intend to pull the user’s eye …. but I wonder if it’s putting the emphasis enough where we want,” she said.

Fading Home Page Slow People Down

Since we were talking design, I asked about that funky fading home page that Google’s been testing. Is that going to become permanent?

“The fading home page is either going to be reformulated or go into a whole new direction,” Mayer said. People visit it more and do more searches because of the change, she said metrics show. However, ironically, the page also slows them down. They take additional milliseconds to act (and for Google, every millisecond counts).

“Their time to first action is slower,” Mayer said. “It’s almost like they’re disoriented, thinking ‘What? Where is everything?”

That brings things back to the search results page. People like patterns. The more a page has an easy pattern that can be processed — the more “rhythmic you can make it,” Mayer said, the faster she says people will be.

So who sees the changes? About 1% to 3% of Google users, who will be randomly selected. The test will probably run for about six weeks. If successful, expect to see the changes — altered to take in account test feedback — show up across Google soon after that.

Postscript by Barry Schwartz: There are some people actually seeing the new user interface in the wild. Here is a screen shot of what people are seeing:

The expected Google Music Search is real, and our Google Music Search 2.0 Launches With Musical “OneBox” story provides the play-by-play of how it works. Be sure to read that! From me, the color commentary out of today’s launch event at Capitol Records in Hollywood. I’ll be live blogging the news.

We’re expecting some musical stars, including guests from OneRepublic, Dead By Sunrise, Linkin Park and Mos Def. I’ll do my best to keep up, given my lack of popular musical culture makes me the butt of jokes to those who know me well. Have mercy, OK?

And it’s 4:04pm, with loud music playing, and a stranger intermix of tech folks and music folks. And nothing happening. So just stay patient. It’s “The Joker” playing, by the way. You know, midnight toker?

OK, a little video and Syd Schwartz, senior vice president of EMI takes the stage. Talking abou the history here in the Capitol tower some of the big artists. Me, I’m just afraid we’ll all die. Because it’s the only landmark building in LA and it always gets blown up in disaster movies.

Now Marissa Mayer from Google. “It’s clear to us that for our users, music holds a special and deal place.” Or a quote very similar to that.

Google mission to organize info. But quickly realized they needed more than web pages to do this. So image search in 2001. Book search in 2003. In 2005, Google Maps to search the physical world. 2007, Universal Search blended this all together.

How’s music fit in. Top 10 searches involve lyrics and musics. Hey, there’s a Hugh Grant and Drew Barrymore movie called Music & Lyrics. It’s awesome.

Marissa’s routine is to listen in the car, memorize a snippet of a song, then going back and searching for it. But that’s not easy. So, “why can’t Google make music more intuitive and easy to find online?” So excited about the new service today (HEY, READ OUR ARTICLE UP THERE IN THE FIRST PARAGRAPH FOR MORE ON THAT). Only service that lets you play an entire song. And Gracenote allows them to do full lyric search [don't ask about all those sites listing lyrics illegally in some cases in Google's regular results].

Now RJ Pittman to demo. Music in his blood, mother is a music teacher. One of his favorite bands is OneRepublic. But is it one word? Ryan from the band he jokes tells him that’s right. But excited you can do two words and find it that way. Now showing how you can get music one click away. Hey, OneRepublic is pretty good. I should download them. From Amazon, cause I like how they’re easy and DRM free. Wait, I mean from one of the Google partners that are involved with it. Don’t worry, see our article, Google doesn’t earn off that itself.

Searches for a lyric from a Police song. No, not that one. You were thinking Roxanne. Not that one. Long lyric, finds a OneBox, plays music. People applaud.

“If you ever wondered what a search result sounded like on Google, this is it,” he says, as he plays Dead By Sunrise.

Marissa’s back up and talking about MySpace and LaLa for live music streaming, music vendors imeem, Pandora & Rhapsody. And Gracenote for Lyrics.

Now Ali Partove SVP of BizDev for MySpace and Courtney Holt president of MySpace Music. Talking how they’ll be able to better get news out about tours. How the social graph can be useful to the music experience. Music has helped MySpace grow and this will help even more.

“Music on Google Search. How F-ing cool is that,” says Holt. Except he didn’t have that dash in there :) Hey, we’re at a rockish event gang.

Now Holt showing popup you get when you click on Google music results, allows you to by the MP3, watch the official video or get concert info. If you clickthru, you get to a dedicated music page at MySpace for the artist. Now he’s giving props to the Google team for building. “In my 7 years in digital music, coming up for an arrangement…” where consumer, artist, rightsholders and many others win is hard, but says Google’s done it. “the main beneficiary is the consumer” getting a better music experience.

Now Bill Nguyen, chairman and founder of LaLa. Or lala, I guess. “We all begin music discovery at the same place, Google Search.” Well no, but OK. And now he’s showing three bands that haven’t gotten discovered more. First is Edward Sharpe & The Magnetic Sharpe. They sound nice. I’ll download them.

Shows how on lala, you can go to their page and see who has discovered them as a fan, see other music that person likes, you can follow folks like Twitter. It’s Twitter for music lovers.

All built on the web because he hates MP3, doesn’t like to organize them. Me, I love them. I want to own my music hardcopy, I guess. But he shows how you can buy the music on their service, what you already own. I assume you can download. I assume you can then put on your MP3 player. Right?

Showing how you can preview a song and buy it if you like it. I kind of do this on Amazon already. But I think you can listen to the entire thing. So that’s cool. Now showing what looks like an iTunes to organize your music. But not probably as sucky as iTunes. And from lala, on the web.

RJ’s back. Goal was to keep the product simple. Marissa taught him sometimes greatest challenge is restraint in features. Make it so if you know a few words, boom, you can get that music in a click. [Hey, what about being able to sing it. Kind of like Shazaam. But there are other services out there like that, too. Let's have that, too!]

When you’re over 25, you stop listening to new music he’s heard. Yeah. [But then when you're past 40, you start feeling like you'd better listen again or you'll be really out of it. Just saying.] But he’s excited this will let people discover more. [Except, I've gotta say, there's not a lot of discovery here. It's all search. It works because you KNOW an artist or KNOW a lyric you heard. It's not suggesting new stuff you don't know].

Now a roundtable. We have Mos Def, joining Wendy Nussbaum of Universal Music Group, Steve Savoca of Domino Recording, Mike Shinoda of Linkin Park, Ryan Tedder of OneRepublic and Syd Schwartz from EMI.

Asks Mike about how things have changed over time. Mike talks about how Linkin Park had a different name, found the name Lincoln Park but couldn’t get that domain. “We thought, this could be an important thing to have a dot.com of our name.” So we literally changed the name of the band to get the URL that would give us the most direct link to our fans. [and yes, I know he also said this yesterday at the 140 Twitter conference. He mentioned to everyone here he told the story there, too].

RJ asks if they thought about getting non-US domain names for the band, like in France. Mike says thought about that.

Wendy now talking about how this will lead people to legitimate sources of music [and I hear Apple thinking now ugh, maybe you know holding all our legit music behind that absurd you've gotta have iTunes to find it AppleWall is something we ought to take down].

RJ asks about the idea of seeing what people share, he’d love to see what Mos is listening to, as Linkin Park and OneRepublic. Wendy says basically sure, always thinking about new ways.

Syd now talking about how he wanted to build out his jazz collection, going to Tower Records and seeing Donald Fagan of Steely Dan. Followed him around as he’s a jazz expert. “So trying to not look stalkerish,” he followed him for 1 1/2 hours. “I discoverd some great stuff .. but I look at what’s been presented here today and think wow, I could have saved myself a restraining order.”

Mos asked about what gets him excited in terms of the web and technology with what he does.

Says sure, for all artists, the internet and Google and YouTube and things like that have been incredible important. And now a billion cameras going off with high speed shutters. Thought we were all digital these days.

“Me, to be perfectly honest, I’m a child of hte 70s, so I walk around every day thinking I’m in Buck Rodgers or Battlestar Galactica … I’m still really getting over the cell phone.”

See now similar to early 20th century with new tech, a wide open field. … “I’m still getting over email. Wow, it’s incredible.” Excited at opp for independents and how quickly they can turn music around.

Steve says excited to have independent have a seat at the table. “This means a lot to us. What we do is niche.” Digital is 50% of their business and primarily get people through word of mouth. “This is a zero friction music experience.” Hear about an artist, hear the artist. With things like lala and OneBox, feel new opportunities to get their music out

“We’ve got to change consumption behavior.” Need environment where people can access these things. When we can expose people to these great services we can change behavior and that the legitimate side of things is where to be.

Ryan talking about how he feels MySpace is what made their breakthrough. He got dropped same day as Katy Perry and Jonas Brothers from the same label. Audience goes wow, because even I know they’re all hot. Thought if he had to make poster and put them up, “I’m screwed.” Found MySpace thought it was perfect, free. Would search for everyone 18-22 on MySpace and emailed them when in town to get them out to concerts. “Were it not for MySpace … that would not happen if it weren’t for technology as it is [people coming out].” For Google, when type in a song until today are bittorrent illegal download sites. [which is kind of bad for Google, because if that's true, well, they're still listing all that stuff]. He’s excited that it literally focuses stuff.

OneRepublic has new album coming. Any new plans in the digital space? Yes, might do something with Twitter. “I don’t know what artists did 20 years ago .. did you rent out the Rainbow Room and throw a kegger.” I’m loving Ryan now, because he’s so totally using this entire new medium. “I kind of have two jobs, writing and producing .. I need things that won’t take up the whole day.” “It’s a blessing and a curse, because we have to be in 20 countries at once.”

Mike from Linkin Park says you’ll be able to see how they make their next album online.

And now we’re going to do Q&A.

How will this change industry in future? Courtney says Google has lots of queries but hasn’t been leading people to a legit experience. This gives consumer more uniformity. It actually gives the rightsholder more control, a way to make money.

Mike from Linkin Park, “I want as few steps as possible between me and that think I’m looking for, the band … I want to be able to find them amidst the noise.” This is what excites him most about this.

Bill talking about how this will let people hear more diverse music. “They’re going to do for music what they did for the web.”

What about outside the US and streaming. RJ says right, only US right now. Focus is to go heads down and tackle this market.

I asked Ryan and Courtney about how they’ve both mentioned there being illegit content on Google, and how that’s not going away, so what do the think Google should do. Or is being at the top of the list enough?

Ryan:

“I’m not entirely sure that it’s Google’s problem … I think it’s a huge step foward … first major company outside of a record label that filters through the noise … again that was the first thing that caught my attention. This is the first internet anything that does that … I don’t know how they did it. You’ve got people way smarter than us [says looking at Marissa and RJ of Google] … You didn’t have to do it. .. as for the fans … clicking on dead links [is bad] … now you’re playing the song two seconds after typing in a lyric.”

Courtney:

“The number one way for the music industry to battle piracy is to make the legitimate image options better … here what Google’s done is made … an extremely convenient experience … also the speed of it … google has had an enormous focus on making it really freakin fast … I honestly believe simply becasue it is more ocnvenient and it’s from a brand that’s more [recognized, I think, he said]” will get it used.

Question on costs. If this generates more listening, more costs? Mike for lala says “more listening is more buying.”

“They don’t want to talk about it … they’re not going to tell you” Mos says, as another question comes up about buying and selling and how the music reps haven’t wanted to get into that.

Mos also impressed that labels seem to have jumped because this is Google [idea that Google is being, versus Napster .. I'll try to find another live blog account that caught this part better]. Live blogging is hard, did I say that?

And that’s it, the announcements all done. Techmeme has massive related coverage, both live plus regular articles that will be coming. Check it out. And rock and roll, people! Heh.

Love it or hate it, the Open Directory Project (ODP or “DMOZ”) always seems to creep into the conversation when we’re discussing links and/or SEO. Check any forum, social news or answer site and you’ll see a wide variety of opinions on the 11 year old directory and how it’s managed.

When talking about the DMOZ two camps typically emerge, those who support the directory and its mission (editors) and those who support getting into the directory (SEOs). While the two sides tend to clash, I’ve found the “directory trust” SEOs seek is the same trust DMOZ editors feel they need to protect. The determination by both groups to satisfy their goals fuels the constant struggle between webmasters and directory.

Since DMOZ is not a search giant, and seemingly does little to promote itself or the core values of the directory, you have to wonder why editors and SEO’s even bother with it. But since they do, I decided to approach DMOZ for input on a number of commonly asked questions and post answers and viewpoints here at Link Week. Nothing earth shattering came out, but I did pull a number of interesting tidbits from the interview and the research I conducted to support my input.

To get things going, I asked my good friend and AOL Manager Simon Heseltine to introduce me to someone at the DMOZ. That led to a meeting with ODP Editor Myron Rosmarin, which led to a dialogue with Founders Rich Skrenta and Chris Tolles who pointed me to Bob Keating, Editor In Chief of the Open Directory Project who graciously agreed to the interview.

Debra: Tell us a little about yourself Bob, how did you get started with the Open Directory Project (ODP) and how long have you been there?

Bob: I have been working on the ODP since I joined AOL in June 1999. Initially, I was brought on to work on a directory solution for AOL Search. I joined the ODP team shortly afterward to help develop the ontology and the community self-governance model. About a year later, the ODP Founders appointed me Editor in Chief.

Over the years, I’ve worked on number of search and publishing projects at AOL. In 2004, I left full-time employment with AOL, and took a position with the Federal government to start-up a new search engine program, but I remained as a consultant on the ODP. Since 2006, I’ve worked in the strategy consulting space, helping Federal clients develop product strategies around search, social media, and web-based services.

But through all these career changes, the ODP has been a constant in my life. For the last five years, my involvement has been more focused on overseeing the community and advising the ODP team at AOL on everything from the project’s history to community interaction.

Debra: Why is the directory sometimes referred to as the ODP and other times DMOZ? Is there a difference?

Bob: The directory’s “official” name is DMOZ: The Open Directory Project. DMOZ means “Directory Mozilla” – the idea was to align the directory with the Mozilla brand, even though it was not actually part of that group. DMOZ and ODP are now used interchangeably to refer to the directory.

Debra: Most of us know that DMOZ is owned/operated by AOL, but the site still lists Netscape as “hosts and administrators”. Who ultimately makes the “big decisions” at DMOZ?

Bob: By design, it is the community that makes the “big decisions.” But in terms of the corporate entity that is ultimately responsible for DMOZ, it is AOL.

Debra: Can you explain the chain of command at DMOZ?

Bob: DMOZ is essentially a meritocracy in which editors are granted high permissions based on their interest and quality of participation. There are two general types of permissions: those that allow one to edit anywhere, and those that allow one to participate in community management. An editor with the former permission is known as an “Editall.” Editalls can edit anywhere are engaged in discussions around taxonomy and the editorial guidelines.

An editor with the latter permission is known as a “Meta Editor.” Meta Editors are community managers and are responsible for reviewing editor applications, investing and resolving abuses, and leading editor discussions. For all intents and purposes, Meta Editors and Editalls are “equal” permissions but focus on different aspects of the community.

The “Administrator” permission is the highest community management permission, and is granted to a few, trusted editors to oversee the day-to-day operations of the community. They ensure that Meta Editors and Editalls are being fair and equitable, and that the guidelines are kept current.

The ODP’s governance model is intended to be self-regulating, so there are checks and balances in place to ensure all topics and all points of view are represented, and to foster an inclusive environment in which any editor who wants to contribute is encouraged to contribute. This model doesn’t always work perfectly, but it has been very successful in creating a self-regulating environment, which actually has less to do with the model and more to do with the extraordinary group of editors who contribute to the directory’s governance.

Debra: How do you respond to the allegations some DMOZ editors accept money for submissions?

Bob:Accepting money for submissions is strictly against the community codes of conduct. In cases where we have confirmed this is happening, we revoke the editor’s account. That said, in more cases than not, the allegations are just that… allegations. Still, accepting money in exchange for submission is a consequence of an open directory operation in a closed community.

As I mentioned previously, our challenge is to create a system that allows Webmasters to contribute to the ODP, rather than feeling disconnected from it, which gives one incentive to abuse the system. This solution involves expanding the ODP’s scope, offerings and participation levels. I can’t promise the solution will rid the ODP of nefarious activity, but I think becoming more inclusive while still retaining the directory’s self-governance model will be a significant improvement.

I think it’s important to note that our editor application review process is very thorough. From a directory quality perspective, the best time to identify potential abusers is before they get a foot in the door. We ask that applicants provide a thorough listing of site affiliations and we use full disclosure (as opposed to the affiliations themselves) as a criterion for selection along with general editing quality of the sample sites they provide. While this may mean that we occasionally reject good applicants, the end result is that we keep out many potential abusers. That’s good for everyone.

We unfortunately do sometimes encounter editors who abuse their editing privileges for personal gain. We have a system of community policing to help weed out these “bad eggs.” The public, as well as other editors themselves, are able to report suspected abuse via our abuse reporting tool. When a report comes in, meta editors investigate these allegations fully and if we find that they have merit, we revoke the editor’s account. In the case that a meta editor is suspected of abuse, the case will be investigated by an admin.

We recently did a blog post about what editor abuse really is and what information we need to have in order to fully investigate it.

Debra: There are a lot of categories at DMOZ without editors. I know there is an open invitation for anyone to apply, but what is DMOZ doing to recruit people to fill the empty categories?

Bob: Even though there are lots of categories without listed editors, anyone listed in a parent category or with directory-wide editing permissions can edit these categories. So, even though an editor is not listed in a category does not mean the category is not being maintained.

We are an all-volunteer force, so recruitment is primarily through word of mouth from our current editors and through data users themselves. The editors reach out to others within their own communities and this has produced tremendous growth in some areas. We also get new editors who find us via the DMOZ data attribution badge on other sites or because they learn about us by seeing our results in Google or another search engine. DMOZ gets hundreds of applications daily, and routinely accepts those most likely to edit well and contribute more than just their own site.

Debra: Yes, I understand category editors can/do pitch in, but when I look at a major category like Real Estate and notice seven of the first nine categories are without editors and one category shows 2007 as the last date the page was updated, I have to wonder what the Directory is doing to keep its results fresh. How can a handful of people in a major category like Real Estate keep that section of the Directory current?

Bob: The date at the bottom of the page can be misleading. It’s not always an indicator of freshness. Some pages are not updated frequently simply because they are directional pages (i.e., they direct users to categories where sites are listed); or because the kind of site listed in the category is so specific that few sites are listed at that particular level. http://www.dmoz.org/Business/Real_Estate/Agents_and_Agencies/ is a good example.

The category description page explains how agency sites are listed. The lion’s share of agency sites are directed and listed in the Regional area of the site, which is where a lot of the editors in this area are spending their time and effort.

Debra: Has there been any discussion about the ODP offering a paid review program?

Bob: This issue has been raised and discussed many times. Paid review really goes against the whole idea behind the ODP. In fact, our Social Contract with the web community takes an especially firm position on this issue.

Debra: Why can’t DMOZ notify webmasters when their sites are included or rejected? Has there been any discussion on being able to pay for this feature?

Bob: Because the ODP is not designed to be a site listing service, creating a notification system has not been a priority. In the past, there was a “check my site status” thread offered via the editor-run public forums at Resource Zone (www.resource-zone.com). It was not hosted or administered by AOL. It was a good faith effort to reach out to the webmaster community.

However, the thread got quite unruly and unmanageable, so it was taken down. Moreover, some editors felt the “check status” thread conflicted with their other editing pursuits. Nonetheless, I can see us adding this as a feature in the future. As with any feature associated with DMOZ, it would be provided at no cost.

Debra: What are the top three reasons sites don’t make it into the ODP?

Bob: They are:

1. The site was submitted to the incorrect category. Editors may move these submissions to the correct category (which can significantly delay review); or delete them from the submission queue.
2. The site is incomplete, under construction, returns an HTTP error, or lacks adequate or unique content.
3. The site’s content mirrors a URL that is already listed in the directory.

Debra: Mention DMOZ to a group of webmasters or read forum posts discussing the directory, and you’ll usually find the negative comments far outweigh the positive. How is the ODP dealing with their legacy issues?

Bob: Webmaster angst stems from the fact that the ODP is not designed to be a site listing service for webmasters. Webmasters have worked very hard to make the ODP work for them, and the editors have worked equally hard against Webmaster tactics that are contrary to how the directory operates. As result, this conflict has created a cloud of distrust and negativity between both camps.

The Webmasters feel shut out of a community that was intended to be open to all types of contributions. For a while now, our challenge has been to create system that allows Webmasters to contribute to the ODP in a mutually beneficial and meaningful way, while preserving editorial quality.

The solution is not as simple as turning the ODP into a submission service or maintaining the status quo. Rather, the solution involves expanding the ODP’s scope, offerings and levels of participation. This is at the heart of what we are working on today.

Debra: It’s great to hear the ODP is working to expand its scope, offerings and participation levels, can you tell us a little more about your plans and when we can expect to see them implemented?

Bob: ODP is committed to expanding its scope, offerings and participation levels, but I can’t share any details with you at this point. When we are ready to announce more details, we will be sure to let you know.

Debra: Do you think people would be so passionate about being included in the directory if it wasn’t used by Google?

Bob: It depends if you are talking about Webmasters or editors. Clearly, webmasters would not care much about DMOZ if it weren’t for its influence on search engines. Editors, on the other hand, have a different perspective. The reasons editors participate in the ODP are as diverse as the global makeup of its participants.

Debra: There was a post on the DMOZ Blog recently where an editor (crowbar) outlined what makes content unique by ODP standards. It listed a number of points but seemed to dwell on the issue of mirror sites, or that “A site should not mirror content available on other sites”. Since this is a strong criteria for inclusion (or not) in the Directory, why does the DMOZ give away its content through the dump program? On one hand, DMOZ admits to deleting sites submitted that don’t have unique content and yet they provide mirror content to anyone who asks. Is this a case of do as I say and not as I do?

Bob: There are two separate issues here. One is content distribution and syndication, which the ODP does as do billions of other websites. Sites that include syndicated content are not considered “mirror sites” simply because they include syndicated content.

The other issue is content that an entity replicates over different branded domains. This is a common tactic in e-commerce, and is the issue the guidelines around “mirror sites” are intended to address.

The interview ended there. Here’s my tidbits and takeaways:

The tidbits

When I heard Bob make this comment:

“the lion’s share of agency sites are directed and listed in the Regional area of the site, which is where a lot of the editors in this area are spending their time and effort.”

The word “regional” caught my ear. I’ve been following Tim Armstrong since he came on board as AOL’s CEO and understand he (and now AOL) have a strong interest in Patch.com. It’s interesting to note Patch.com is a regional, community specific platform showing news and events from specific cities and towns. Seeding Patch.com with regional results from a respected directory would make a lot of sense, so if you’re bricks and mortar based, now might be a good time to submit your business to DMOZ.

The second tidbit worth noting, is the comment about the notification service. Notifying webmasters why their sites aren’t being added to the directory would go a long way in eliminating the frustration many feel about the ODP; after all, education is preferable to being ignored. I sincerely hope this project moves along much faster than the DMOZ 2.0 project they dropped hints about back in June 2008.

The last and most notable takeaway from this interview, IMO, is the response to my question on why sites don’t make it into the DMOZ. Bob’s answer is informative and also very unsettling because it speaks directly to what I feel is the core problem with the DMOZ – a lack of editors.

Here’s what he said when I asked “What are the top three reasons sites don’t make it into the DMOZ?”

“The site was submitted to the incorrect category. Editors may move these submissions to the correct category (which can significantly delay review) or delete them from the submission queue“.

I’ve spoken to many DMOZ editors who all say the same thing, they delete submissions made to incorrect categories rather than send them along. Why? I’m told it’s because so much of the directory is without editors and/or because they have the authority to do so.

Hmm. This attitude is interesting especially since the DMOZ states “fairness and objectivity prevail here” in their editor requirements. It doesn’t seem “fair” or “objective” to simply delete a submission added to the wrong category but hey, that’s the way things go at the DMOZ. Say anything and even top management is quick to point out “the ODP is not designed to be a site listing service for webmasters”. I think you’ll find a lot of webmasters support that statement and want a quality DMOZ maintained, they just don’t always get it right when they submit. Submitting your site to the wrong category should not preclude you from being added to the directory.

One of the reasons for doing this interview was to find out what the DMOZ was going to do about recruiting editors to fill its very empty ranks. While Bob reaffirmed the DMOZ’s commitment to quality editing, he didn’t address the issue of recruitment, even though I asked the question twice.

How can the directory maintain quality content with so many categories missing editors? Case in point, when I view the page dedicated to the hot topic of H1N1/Swine Flu, see no editor and note the page was last updated October 18, 2009 I wonder if the DMOZ is really a serious search source.

Add to it, I don’t see popular sites such as the Mayo Clinic, the World Health Organization, MedicineNet or FluView listed and now I’m also wondering about the ability of ODP to provide relevant information. It’s not hard to list the top health sites on the Web for the term H1N1/Swine Flu, but it’s  impossible when you don’t have a editor working the category.

Yes, yes – I know section editors can and do come in to edit but they’re obviously not doing that here, are they? For topics in the news or representing financial/health issues, every effort should be made to fill those categories with qualified editors and keep the category updated. To do anything less is a disservice to the public and the directory.

I sincerely hope DMOZ doesn’t become invisible like the Great Pumpkin, as it has been an integral part of the search industry for 11 years and deserves respect for its contributions. A hand-edited directory of 4.5 million websites is an accomplishment no one else can claim and I support the stringent admission standards they have in place. But I also hope the directory makes every effort to utilize the vast resources AOL has to recruit quality editors to its empty categories. The H1N1/Swine Flu category is a classic example of how out-of-date the directory is and how important editors are to keeping it current. I believe once editors are in place, many of the other issues will take care of themselves.

Hey AOL, this is no trick, we want DMOZ to be our favorite treat! Are you listening?

Microsoft’s Qi Lu announced two new partnerships today at the Web 2.0 Summit in San Francisco. Here’s a transcript of his chat with Tim O’Reilly.

Qi Lu

I was at Yahoo for 10 years, and I always told Jerry, after 10 years I’m going to try something new. I went to Microsoft because I wanted to have an impact.

Search is about computationally understanding user intent. Understanding their interests and needs. That’s “absolutely” our long-term goal – to “build a mind-reader.” Once you understand user intent, you can figure out different solutions. There are many queries where images are best at fulfilling intent. Sometimes videos are best at fulfilling intent.

Ahead, you have Facebook and Twitter – particularly Twitter. There’s a strong velocity of things flowing through that will enable people to get answers.

Yusuf Mehdi to do a demo now.

Bing launched about 100 days ago and off to a pretty good start.

People will start to ask more complicated questions. To answer that, you need better access to data. Starts demo of Bing Wave 2. Shows recent photo search tool. “It changes the way you do search.” Shows example of search for “top iPhone apps” — this has “really resonated with consumers.”

Second area is about real-time, hot information. Two announcements:

Partnership with Twitter. Been working on it a long time. We will get access to all public Twitter information in real-time. “We’re gonna do some really exciting things.”

Other partnership is with Facebook to get access to all of their public data, and that will come at a later date.

Now showing example of Twitter integration. Shows full, real-time feed. Looks like existing Twitter search, just wrapped in Bing interface. Will update automatically.

Bing applies “best Match” technology to tweets: de-duping, then focus on

quality – look at who’s tweeting and assign soicla relevance score popularity – look at the caption – length of comment, links, etc., affect quality and relevance usefulness – number of retweets affects results

Then we apply spam filtering.

Bing also shows hottest topics on Twitter – a tag cloud. Looking at tweets related to these items – Bing takes most popular links and assigns its relevance algorithm, then shows the hottest links, too.

Pick a topic like “Yankees” – Bing shows most recent tweets, then shows most popular links. Bing pulls out landing domain from shortened URLs.

This is beta code. Sean Suchter working on it at headquarters. Should be available shortly at bing.com/twitter.

Demo over. Qi Lu speaking again.

We don’t disclose financial terms with Twitter. “I don’t know specific length of agreement. This is a start.”

The key thing is to have great teams, great technology, and keep the pace of innovation.

Audience questions now.

Qi: In a real-time corpus like Twitter, there’s a lot of signal velocity. The Twitter corpus is still evolving. But based on what we see, you have enough signals to have meaningful content. A lot is meta content that refers to other content on the web. But this is just the beginning. The future is very compelling.

Are you keeping the Twitter stream? Are you archiving the firehose?

Qi Lu: Don’t want to answer because I may not be accurate.

The Twitter deal is non-exclusive.

Is Twitter allowing search engines access to protected tweets or not? Not, Twitter tells me, though the company probably needs to do a bit more to prevent this type of confusion in the future.

The LA Times reported yesterday about a “Twitter hole” that it believed allowed Google special access to protected tweets, tweets made from Twitter accounts where owners have deliberately chosen not to have their tweets be made public.

Not so, said TechCrunch. The so-called protected tweets that the LA Times was finding in Google looked to be those made from before particular account holders locked down their accounts.

I checked with Twitter and got back the official word from their press office:

The TechCrunch article seems to sum up the confusion pretty well. It seems that the LA Times piece references tweets that were public but later the user protected the account, thus all subsequent tweets are private along with the profile. The tweets prior to that time cannot be un-cached.

Google has not been given a key to the castle…so to speak.

I’m good with this answer except for the word I’ve bolded — that formerly public tweets cannot be uncached. That’s incorrect.

Let’s take an example. Let’s assume you started your Twitter account in March. You started tweeting publicly, then in July decided to be private. Twitter doesn’t try to protect any of your past tweets. In fact, it’s pretty clear about this in its help page on the topic:

If you have a public account and you protect it, all updates after the time of protection will be protected. Your profile will only be visible to approved followers, and existing followers will not be affected.

Please note that tweets from protected profiles will not appear in search results. People will still be able to find your account using the Find People search tool but only people you’ve approved to follow your account will be able to see your tweets. Also note that any tweets posted while your profile is private will remain private indefinitely, and tweets posted while your account is public will remain public indefinitely

But Twitter could try to protect those formerly public tweets. As best I can tell, if you lock down an account, Twitter does make ALL tweets (formerly public or not) inaccessible to everyone accept those the account holder has authorized to see them. That includes search engines like Google or a Twitter-specific search engine like Topsy.

Well, if Google can’t get in to tweets after an account has been protected, why does it show some? And why does Twitter say this will happen?

Google seems to rely on the last information for a tweet that it could see. So you tweeted something in March. Google sees the tweet and records it. If in August, you protect your account. Google tries to revisit your tweets as it does with any web page, to make sure it has fresh information. It can’t get to any of your tweets now.

The ones from August, it never saw them, since they were never public — so it doesn’t list them.

That tweet in March? It keeps showing the information from the last time it saw it. And apparently, it will keep doing this for weeks or months.

Google didn’t send me a comment about this (I did ask, and I might get one later today). But that’s just how I know Google works and can see it specifically working with some protected tweets I investigated today.

As for Topsy, they told me:

Topsy only displays tweets that were once public. The refresh button will make them vanish if the account is now private.

Back to Google. Eventually it should update its old copy of the tweet with what it currently shows to non-authorized visitors, a message that says “This person has protected their tweets” (you can see this for millions of people on Google now).

Twitter could speed that process along by explicitly blocking tweets from a protected account with a meta robots tag configured to remove the page from the index entirely and from cached copies being allowed (the NOINDEX, NOARCHIVE commands).

That wouldn’t guarantee that formerly public tweets are all taken private, of course. Once something’s put out on the public web, it’s very difficult to pull it back. But it could help and seems an easy enough change to do.

If you have a protected account, also keep in mind that those who follow you might retweet what you tweet to the world. If you’re that worried, make sure you pick your followers carefully and regularly keep them informed that you don’t want things retweeted. Otherwise, be prepared for your private tweets to leak out.

For more about search and tweets, see my What Is Real Time Search? Definitions & Players post which cover some ways to make use of Google and its search options feature to drill-down into tweets.

Google’s Sidewiki is here and it’s not hard to see that it could be big, that it could change the way we use the internet. However, it has some frankly scary implications for website proprietors. People can visit your pages and leave messages, whether you want them to or not.

Dell Homepage with Google SideWiki – Image credit: Flickr

I think the best description I’ve seen to date was on Econsultancy, describing it as “a way to graffitti websites.”

How can we web marketers respond? Have we lost control of our own websites and those of our clients? There have already been several instances of rude, insulting comments being left on the pages of major corporations.

So how can we make the best of Sidewiki? Here are my thoughts, and I welcome your comments. Haven’t yet seen Sidewiki? You’ll need to install the Google Toolbar and enable Sidewiki to do all of the things I describe below.

Leave a page owner comment

Sidewiki allows brands to leave special Sidewiki entries on their own pages, entries that are distinguished by a green background.

These entries rank above comments made by other users. They are another way of engaging with your audience via a platform they have chosen and can be really useful.

You could leave some general information about your organization, leave contact details for your customer services department, encourage people to visit your forum or blog, even include some comedic lesser-known facts about your business.

Anything you do engages with readers and shows you aren’t afraid of the power of Sidewiki.

Why not regularly update your Sidewiki with new information, links to press releases, podcasts, blog posts and so on. View it as a chance to speak to your visitors in a whole new way, rather than open season on your site.

Know your rights

Of course, having said that, you do have rights and Google has to recognize them. It would be unfair and illegal to introduce a tool that allows competitors to spam your pages or trolls to leave slanderous comments dotted about your website.

There are some raised eyebrows over whether Sidewiki is even legal and Google has to show some concern over spammers, flamers and so on. It has a ban process that allows people to complain about content.

For example, one of the first Sidewiki comments I saw was on the Daily Mail website. It berated the content and linked to the Guardian.

Now it’s gone and there’s a message from another user that explains: “I reported your post, using ‘other’ as the reason and explained you were simply redirecting traffic to competition.”

While I enjoyed the original description of the site, it is good to know that there’s a simple procedure for complaining about posts.

Don’t be afraid to use this facility in the face of spamming, slander and redirecting.

Be helpful

Wouldn’t it be nice if everyone was nice? Google has said repeatedly that the purpose of Sidewiki is to allow people to “contribute helpful information next to any webpage.”

Its vision, it says, is for a tool that allows people to share useful information and insight. Perhaps doctors can leave informative comments on websites advertising treatments, maybe finance experts can proffer guidance on how competitive a credit card is next to the page advertising it.

While people will try to exploit Sidewiki, others will use it to be helpful and social—just like Twitter, Sphinn or any of the other social tools we’ve seen.

So be helpful. Leave useful comments (not advertising) across industry pages and make sure people know where you’re from. Sidewiki could be yet another way to enhance your online industry profile, which can only help your business.

Offer alternative feedback options

Inevitably, at some point someone will be less than happy with your service. Make sure you give customers plenty of ways to offer feedback to you that aren’t Sidewiki.

Ensure you leave contact information on your site, don’t block blog comments, add a feedback form… Do everything you can to ensure people bring their complaints to you rather than scribbling them on the sides of your web pages.

Of course, you then need to take action to make your client happier with your business—otherwise, frankly, they have every right to leave comments for others to see.

Respond to negative criticism

I hope Sidewiki doesn’t just descend into a maelstrom of abuse and criticism, but it’s sensible to plan for dealing with any negativity. You should treat positive and useful comment as a bonus. However, it’s really important for any business to respond to negative criticism. Do you search Twitter for mentions of your company? Do you respond when those mentions are negative and try to resolve them? Then you’re a fine example of a company that listens and acts, which people respect.

Do the same with Sidewiki. If there are legitimate complaints about your product or service then address them. Your business will improve and customers will know you care.

Want to learn more about how Sidewiki works and how to use it effectively? See Danny Sullivan’s Google Sidewiki Allows Anyone To Comment About Any Site.

Oh, that this weren’t true. Last week, Yahoo made news by disclosing that it had quietly dropped support for the meta keywords tag. As a long time hater of that tag and the insane questions it has produced, I was thrilled! But today, I see conclusively that Yahoo still supports the tag.

The test was simple. I placed a unique word in the meta keywords tag on the home page of Search Engine Land. This word — xcvteuflsowkldlslkslklsk — generated no results on Yahoo when I looked earlier this week. Today, when I searched, it brought back the Search Engine Land home page. Thus, Yahoo indeed indexes the content of that tag. (And to be clear, I looked before writing this article. In short order, this article itself, along with others, will appear because they’ll make use of that word).

During the session last week at SMX East, when Yahoo said it no longer supported this tag, several in the audience said they didn’t believe it. I was kind of struck. You’ve got a search representative flat-out saying they don’t do something, but no one wants to believe them? How things have changed. Sure, I can see distrust on some controversial issues (such as whether Google really does not count nofollowed links out of Wikipedia). But why would Yahoo lie about something like meta keywords support?

To be clear, I don’t think Yahoo was deliberately lying. The representative was probably confused in some way. Similarly over at Bing, despite them NOT supporting the tag (it’s not mentioned here) and never having done so since they launched their own search technology, they recently blogged much advice about using the tag.

As I commented about this:

That reads like someone got a copy of really old SEO advice and decided to put it out there regardless of what Bing actually does. I mean, my head hurts, but not everyone cared about commas or not. And no one had this 874 character limit. I mean, if you went over, it was no big deal. And the don’t repeat more than 4 times? According to what. Microsoft never, ever had its own guidelines like this.”

It’s a good reminder to the search reps. In many ways, you occupy god-like status on issues relating to SEO. Everything you write, everything you say will be fully believed by some. And if you’re not correct, you’ll confuse people and cause others to lose faith in you. If you don’t know, don’t say — or qualify: “I’m not sure” or “I’ll check on that.”

Postscript: Yahoo’s sent me this:

What changed with Yahoo’s ranking algorithms is that while we still index the meta keyword tag, the ranking importance given to meta keyword tags receives the lowest ranking signal in our system.

Words that appear in any other part of documents, including the body, title, description, anchor text etc., will take priority in ranking the document – the re-occurrence of these words in the meta keyword tag will not help in boosting the signal for these words.  Therefore, keyword stuffing in the keyword tag will not help a page’s recall or ranking, it will actually have less effect than introducing those same words in the body of the document, or any other section.

However, when no other ranking signal is present, unique words that only appear in the meta keyword tag section of documents can still be used to recall these documents.

Aardvark has redesigned its site and repositioned somewhat as a “social search engine.” The site can be accessed directly, via IM, Twitter and the iPhone. You can also sign in using Facebook Connect, which sends your activity on Aardvark back to your news feed on Facebook.

What Aardvark is trying to do is not new — there have been Q&A sites for some time — but it’s trying to execute as a better version of the concept of peer-to-peer advice or answers. Once could also call this “human powered search.” Services like kgb and ChaCha also try to do something similar but with trained “guides” or agents rather than one’s network or friends of friends. Yahoo Answers is arguably the incumbent in the space.

Aardvark’s IP is all about question routing and management. I’ve been using Aardvark off and on for a few months and it generally works well and delivers answers (up to four) in a few minutes. The most compelling incarnation of the service is on the iPhone in my view.

The big challenge for the service is to prove that it can deliver more trust, precision or value than a Google search.

I guess Derek Powazek isn’t done with attacking SEO. Now he’s published an SEO FAQ page which, sorry, really doesn’t provide much FAQ about SEO. So what the heck. I’ll deconstruct it. Be sure to also read my previous post, An Open Letter To Derek Powazek On The Value Of SEO.

What’s SEO?

First, any good FAQ ought to define what the subject is, right? So what’s SEO? Turns out, we have an SEO page here on Search Engine Land that explains this succinctly.

SEO — search engine optimization — is the process of getting traffic from the “free,” “organic,” “editorial” or “natural” listings on search engines. All major search engines have such listings, where web pages, web sites and other content such as videos or local listings are shown and ranked based on what the search engine considers most relevant to users. Payment isn’t involved, as it is with paid search ads.

You can read more on our page, along with some links to past coverage, a guide from Google and additional information

Notice in the definition above, I’ve mentioned things like videos or local listings. Search engines, and SEO today, aren’t just about web pages. That’s why when Derek rants that SEO is just some web developer’s job, he exposes a real lack of knowledge of the current state of SEO itself. In turn, frankly, he’s not giving out good advice.

SEOs Are All Scumbags

You know, I went to the gym today. Exercised with a personal trainer. All that trainer did was do stuff with me that should be common sense if you read a book, looked at the instructions on the machine and so on. Why isn’t he some scumbag that’s ripping me off? He’s a professional. He knows exercise intimately, in a way that I do not. It is his job to be an expert in this area. I want that expertise, someone who is totally dedicated to that area, not someone who dabbles in it.

SEO is a profession. Companies from MTV to the New York Times to the Wall St. Journal to Yahoo, to name only a few, employ full-time people who are responsible for SEO (and they aren’t scumbags, either). These companies have some of the best content in the world. And yet, they can still have major issues in how their sites are built or written or constructed that prevent them from doing well in Google or other search engines.

These SEOs, by the way, struggle with web developers who “think” they know SEO but don’t. Web developers who think that despite what an SEO tells them, a 302 redirect is the way to go. And thus the International Herald Tribune loses thousands of links because who wants to trust the scummy in-house SEO, right? I’ve got story after story of web developers and designers who think they know SEO but don’t, who cause major problems for web sites, and yet NO ONE ever writes a blog post blasting them.

There are bad SEOs. There are scummy people who there who spam blogs, who try to go black hat against search algorithms, who figure all’s fair in the fight for traffic. There are also bad people in any industry out there. You don’t count out an entire industry with helpful people because of your outdated ideas of how things work on the web. Especially if only deal with a particular type of web site (Derek doesn’t, for instance, seem to deal much with shopping sites, as best I can tell).

Deconstructing Derek’s FAQ

With that setup, let me go through the “FAQ” points Derek’s put out there and comment on them:

I publish a magazine and I know a lot of magazine publishers. And they are forking over embarrassing sums of money to charlatans who say they can raise their search engine rankings. These magazines can barely pay their writers. That’s wrong and it has to stop.

If you’re a company that’s about to pay some SEO expert, please, I beg you, take that money and hand it to a talented writer or competent web developer instead. It’ll be much better spent.

If you’re a company that hasn’t reviewed your site for SEO aspects, I beg you, consider this. Unless you like the idea of potentially wasting money, it pays to ensure you’ve done the basic things out there that will give you traffic for free. Maybe everything’s all perfect already, that your web developer who also knows some things about SEO has it all down. But it’s cheap insurance for many sites with a budget to ensure they’ve implemented SEO properly. Or, you can buy AdWords and pay by the click for that traffic from Google.

As for charlatans out there, yes, there are some. Just like there are some bad writers, bad designers and bad web developers. Ask for references. That’s not rocket science.

But I use SEO for good. Then you’re called a Web Developer. Good web development includes using proper formatting (like putting headlines in H tags) and understanding how the web works, search engines included. Valid code also has the side-effect of making your pages more accessible for your users, which is the point. Making your pages more accessible to robots is for robots.

Hey, I can fire up WordPress using the Thesis theme and make a nice site. Am I a web developer now? No need to pay some web developer hundreds of dollars per hour. I’ll do it for you as well.

Web development is not SEO. Good web developers will understand the fundamentals of SEO, in terms of good site architecture, crawlability and so on. But they probably won’t be ensuring that authors of a site are tapping into keyword research tools to ensure that when they write an article, they’re using terms that an audience seeking that article might use.

More important, few of them are dialed into how to handle giving Google and others a shopping feed. Or a feed of real estate listings. Or the completely separate ranking aspects that impact YouTube (the world’s second most popular search engine). Are they putting out a full-feed that Google Blog Search prefers? Are they checking that the URL shortener you use on Twitter spits back a 301 rather than a 302 redirect or worse, frames stuff up via a 200 code?

This isn’t 1999. This isn’t put stuff in a H1 tag and you’re good. This isn’t a world where we lack important tools such as the ability to feed sitemaps or even today, a way to see pages exactly as Google sees them. SEO encompasses any type of search engine dealing with any type of content.

[Insert irate defense of SEO here.] You sell SEO, right?

Yes, some of the people who are defending SEO also either perform it or are related to the industry. And some of those same people will readily admit that the industry has a terrible reputation. But then again, they actually KNOW SEO. They’re responding because unlike people like Derek or others who launch rant posts like his every six months or so, they deal with confused people all the time. They pick up the messes that web developer make. They solve the real problems out there that just being “real” won’t help. The don’t deserve to be written off. They actually deserved to be listened to, because they have lot of valid things to say.

If you’re so smart, prove it. I’m not that smart. But I can say that I am Google’s third result for “Derek” (I was number one until Wikipedia came around). And after less than 24 hours, my post about SEO is the ninth Google result for “SEO”.

Well, if Derek’s SEO strategy was to find people and convert them into sales for those seeking “Derek,” congrats. But actually, part of SEO is understanding what a target audience is after, in terms of your content. And I don’t care who are. Unless you’re doing a personal blog with absolutely no particular audience in mind, you do have a target audience. You’d better understand how that audience might be seeking your content. Otherwise, you actually might not get found. Good SEO starts with good keyword research or at least an awareness of it.

As for Derek ranking of SEO, welcome to the world of “query deserved freshness,” where Google will reward fresh posts with a relevancy bump. Does your web developer understand that? Because if they don’t, then when their boss starts complaining in a week how your great ranking just went away, they might start messing with your site to regain it — and make things worse.

How did I accomplish these magic SEO feats? Exactly how I said I did: Make something great. Tell people about it. Do it again. I’ve been doing that (or at least trying to) since 1995.

In this case, I wrote a passionate post. I posted it to Twitter and Facebook. And I sent one email to a friend about it. That’s it!

Sure, Derek is passionate about an industry that has a lot of haters. Guess what. I’ve been writing about SEO since 1996, telling people the good ways to do it and why it helps. And you know, it HAS helped lots of people.

If SEO is just all snakeoil, why am I still here? Why is SEO still here? Derek is hardly the first to have a rant. I’m easily into double-digits of rants like his I’ve read over the year.

The answer is that it ISN’T all snakeoil. SEO really is and can be useful. It’s just a pity that more people who do get helped by it who aren’t SEOs themselves don’t speak up for it. But here’s another tip. If your company does well with SEO, you usually don’t want to talk about it — because you’re hoping your competitor is listening to people like Derek who say SEO is all bullshit.

SEO is needed because of bad web design. Wouldn’t it be better to make competent websites in the first place?

Yes, it would. And when the miracle happens when all web developers are doing that, please let us know. In the meantime, perhaps instead of antagonism, we can talk about ways that a good web developer can identify a good SEO to work with? And also understand that even then, SEO extends beyond web pages these days.

It’s easy for you – you’re an expert. I’m using WordPress (which is free software) and the DePo Skinny theme (which I designed and released for free). You can download both of those and have a site just like this in a few minutes. No need to pay anyone for SEO. Then comes the hard part: Spend over a decade making things. If you do that, you’ll be an expert, too.

So just be an expert in selling ranch fencing, and you’re good. Or one on hazardous waste disposal in your local community, and you’ll rock. Or toss up a blog about your local locksmith site, and you’ll be cooking along. Or not (see Searching For Small Businesses, Coming Up Frustrated).

Passion is great. Wonderful content indeed rocks. And even “boring” businesses can tap into viral popularity. But you also have some boring SEO stuff you probably want to attend to, as well.

[Insert personal attack here.] I may have tarred a so-called “industry” but I didn’t attack anyone personally.

Really? Let’s review some statements:

Search Engine Optimization is not a legitimate form of marketing.

If someone charges you for SEO, you have been conned.

A new breed of con man was born, the Search Engine Optimizer.

SEO cockroaches employ botnets, third-world labor, and zombie computers to blanket the web with link spam.

SEO bastards are behind worms that attack blog services like Blogger, WordPress, and Movable Type.

It’s a game, and every link is a score for the SEO jerkwads and their disreputable clients.

Yeah, that’s a personal attack. Derek is calling each and every SEO out there someone who is not legitimate, someone who cons, someone who apparently runs botnets and worms and only has disreputable clients.

None of that’s qualified. He points it at each and every person who specializes in SEO. Part of me is amazed more people haven’t emerged to call him out on this type of hateful, unfair and inaccurate attack. But part of me figures these days, plenty of big voices hear SEO and either don’t want to speak up for them or based on their experiences as blogging rockstars think that everything that works for them must work that way for everyone else.

This may be obvious to you, but it’s not to everyone. You’re right. I regret saying that this stuff was obvious without explaining what I meant. Here’s what I meant: Good SEO techniques are just good web development techniques. They should be obvious to anyone who makes websites for a living. If they’re not obvious to you, and you make websites, you need to get informed. If you’re a client, make sure you hire an informed web developer.

So I heard Chris Brogan speak recently, and he joked that his book Trust Agents (cowritten with Julien Smith) was just full of common sense. Which is true. But what’s common sense to someone who knows their field is uncommon wisdom to those who don’t. Some SEO techniques are also good web development ones. A good web developer should know them or should educate themselves. But plenty do not. Or plenty assume they know it all when they do not.

CLIENTS: If someone approaches you about optimizing your search engine placement, they’re running a scam. Ignore them. If your site isn’t showing up in Google, fire whoever is making your web pages and hire someone better. Sign up for social media services (Twitter, Facebook, etc) and participate there. Pay for quality writers and designers – that’s what will actually raise your ranking in the long term.

Agreed, if someone approaches you out of the blue — you get that email offering to do some analysis, etc. Ignore it. Might not be a scam, but it is generally not a sign of a company that’s in demand because of a satisfied client basis. As for firing your web page builder, um — maybe. Or if you like what they’ve done with the site, and SEO might be able to work with them and allow you the best of both worlds.

WEB DESIGNERS: Learn to code your own pages. If you can’t, hire someone who can, and listen to them when they tell you why putting all that text in an image is a bad idea.

Those people who can, by the way, are called SEOs. You know, those scumbags you shouldn’t be listening to.

WEB DEVELOPERS: Educate your designers about proper web development. Educate your clients about how the web works. Follow Google’s advice. Read A List Apart. Writing good code won’t just help your Google rank, it’ll make certain your site is accessible to screen readers, mobile devices, and all the browsers out there.

Well, I don’t regularly read A List Apart, so I can’t comment on the quality of the SEO advice over there. But yeah, read Google’s. In particular, read their SEO starter guide here.

You might take in a conference or two. At our SMX East conference last week, we had an entire track devoted to technical SEO and developer issues. Want more? Jane & Robot started last year is devoted to developer & SEO issues, and they have events.

You can check out topics here on Search Engine Land, such as our 100% Organic column and our Google SEO, Microsoft’s Bing SEO, Yahoo SEO and How To SEO sections.

SEO SPECIALISTS: If all you do is SEO, you need to expand. Hire a visual designer and some kickass coders and become a real web agency. Start making sites good from the get-go instead of cleaning up other people’s messes. Besides, if all you do is SEO, your days are numbered. Social media is rapidly becoming much more important than Google. (Number one referrer to my site this week? Twitter.)

Again, this isn’t 1999. There are plenty of agencies that offer SEO services. Some of these were pure SEO companies that grew. It’s also no newsflash to many good SEOs that social media has grown in importance. In fact, the real news seems to be that Derek doesn’t realize they already know this.

GOOGLE: Would you look at all the crazy you’ve created? You could fix this by making your engine more inclusive of websites with common mistakes, introducing some randomness to the order of results (why should everyone’s results be in the same order?), and unforgiving punishing the businesses that have sprung up to exploit your popularity. Get on it.

Yes, they do. That’s one reason why Derek’s ranking for SEO at the moment. It’s also the reason that plenty of people like him can say you don’t need to do SEO. Because for lots of them, especially if they have promient blogs, Google does do the right thing.

But it’s not perfect, and you can help it. That’s why over the years, it has rolled out a ton of tools at Google Webmaster Central that any good SEO will know about.

Complaining that Google should just “fix” everything also reminds me of web developers who have long assumed that problems with search engines and Flash would just eventually go away, so they didn’t need to do anything. I’ve heard that for over 10 years. Despite advances, the basic advice remains the same. Try not to lock your content up in Flash.

When developers will test their pages to make sure they work in Internet Explorer, Firefox, Safari and Chrome, not testing against the biggest browser of them all — Google — is foolishness. What Derek’s written above is the same as complaining that Internet Explorer has some bug that’s screwing up his web pages, and rather than fix it, he thinks IE should solve the problem. Sure, IE should — but in the meantime, you workaround it.

Postscript: Let me make clear my intent in this piece isn’t to attack Derek personally. I have issues with many of his statements. Since these statements seemed based on his personal experience, it’s difficult to avoid addressing him on this by name. I’ve read through this piece again, and I really don’t see a personal attack him Derek. On Twitter, in reference to his original post, I did say things like:

yes, let’s make heroes out of people who rant ignorance & harm others who do good work. hurray! (here)

& sadly plenty of SEOs provide good non-sleezy, useful, legit, google-backed services but get tarred with usual ignorant brush (here)

see my tweets earlier today. it’s just the latest in years of ignorant slamming of SEO but (here)

I didn’t mean a personal attack by calling what Derek wrote a rant. That’s just a fact. It was. He called it the same himself. Heck, I rant all the time. There’s nothing wrong with a good rant, though I prefer one that’s also thoughtful and backed up by facts.

That’s where the ignorant part is. I don’t think Derek is ignorant or stupid in general. But I do think much of what he wrote seems to be ignorant of the current state of SEO. Ignorant as in not aware. A better choice of word, which might not have seemed perhaps personal, would have been uninformed.

Anyway, hopefully Derek and I will talk more about this tomorrow rather than playing dueling posts. My intent was never to attack him as an individual, but I certainly wanted to raise some awareness and balance to the issues he put out there.

Derek Powazek launched an attack on SEO yesterday that really said nothing that others haven’t ranted about before. I’ve responded to many of these attacks over the years in hopes of educating people about mistaken assumptions. I’ve largely given up. But I figured this time I’d give it another go with some personal illustrations I’ve encountered recently.

In particular, rather than do an article to counterbalance Derek’s post, I started to write him a private email. But as I composed that, I felt it might better illustrate to everyone why SEO is indeed a legitimate form of marketing and those who provide the service are not all “scammers” who are out to “con” you.

So Derek, I saw your rant, and it was disappointing. Your post was based on “14 years of hits and misses.” Well, my response come from my own 14 years of covering search engines. Of having answered feedback from hundreds of people. Of having talked with hundreds of people personally. Of understanding that the “you just build it; you just put it out there” approach to search engines, sadly, doesn’t always cut it.

Let me be clear. I totally agree with your core advice. Build a site for visitors. Have great content. These are the keys to success, not just with SEO but with anything you want to do. In fact, we just had an article on our site here reinforcing this.

Still, sometimes people have problems. And the stuff that you think isn’t rocket science — that anyone knows — is indeed a mystery to others. They want help, and sometimes they can’t find that web developer who also understands SEO issues. In the same way, you sometimes don’t find web developers who are also designers. Or designers who understand conversion issues. Or conversion experts who understand web development.

Let me tell you some stories.

A Mother Who Sells Homes

Two weeks ago, our local elementary school had a mixer for parents. I was talking with one woman who asked what I did for work. “I write about search engines,” I told her. That led her to asking if I know about how people get found on Google. Yep. So she started asking about her local real estate site, and how she might market it, things that she might do.

That’s a loaded question. See, for you, she just needs to believe in the real estate she’s selling, then tell her friends with personal notes, get out on Twitter, find places where her community congregates and “be real.”

But to really be real, let’s remember that she’s selling real estate in one of the most competitive areas of the country, Newport Beach, California. Her friends aren’t all going to buy homes she’s listing. Her “community” congregates on Google and does things like type in “newport beach homes for sale.”

To succeed in attracting that audience, she should have a great site and great content — agreed. But does she have individual listings? Then she probably needs to kick them out into Google Base, in order to fully be listed in Google. Does your mythical web developer deal with Google Base much? And where’s her web site now? Is she running it off Blogger? Using her own domain? These have impacts on how both the search engines may see her as well as how she’s perceived.

Does she have a blog in addition to a main site? That has an impact. Has she considered some unusual, creative ways to create content around real estate in her area, perhaps some catchy link bait, which may pull in the links she needs to rank better (which, by the way, is a recommended Google practice).

Does she have a local office? If so, has she claimed her listing in Google Local? If so, has she updated her title to reflect that perhaps she has “newport beach homes for sale?”

This is all SEO. It’s not your father’s (or mother’s) SEO that you rant about, the keyword stuffing, the link drops (none of which is best practices SEO anyway). But make no mistake, it’s SEO.

SEO & Baseball Practice

Later that same week, I took my son to baseball practice. I also took my computer, so that I could finish up work on an article and have the weekend clear when he was done.

Another father came over to me, asking what I do. “I write about search engines….” Which as before, led to the questions about how his company might show up on Google. Did I know much about that?

Sure. And since I had my mobile broadband card with me, I fired up his web site. Very sad. Same page title on every page. No keyword research employed, to think how people might be seeking out the industrial shipping cases they sell. Long, dynamic URLs that might pose indexing issues.

Where do you start with someone like this? In your world, his company should just “be real.” He sells industrial shipping cases. How real do you want to be about that?

His “community” are the people who realize they need a case like he sells and go online to places like Google and start searching for the products. And his pages are NEVER going to show up, because there’s nothing unique about his site and he had basic SEO errors that haven’t been dealt with.

If all he did was change things so that his page titles were different, he’d pull traffic. I know, I know — that’s so obvious. But it is NOT to him. His job isn’t to do web development. He’s not you or I with 14 years of having learned all this stuff along the way. He actually deals with things like ordering the products, overseeing workers and doing an array of offline marketing.

Meanwhile, his web developer clearly did NOT have any SEO thoughts in mind when building the site. That’s all too common. If I wanted to be snarky, I could do an entire post on why web developers are a waste of time and you should just employ SEOs who can also build web sites. But the reality is that a good SEO (and there are some) working to help direct the web developer could solve the site’s problems quickly. It’s called teamwork, and it’s awesome when it happens.

Mommy Bloggers & SEO

Over the weekend, I had a group of mommy bloggers over at my house. Not because I’m a mommy blogger but because my wife is a mommy blogger. The group was here because they’re all involved in a new project for mommy bloggers in Orange County.

My participation was to show up at the end of the meeting, not to say anything but just because I was coming home. I caught some of the closing conversation, and a question about Google came up. How was the search engine going to deal with their content, if these mommy bloggers put the same article on a publishers site as well as their own sites.

Answer? I don’t know. Google might decide to favor the publisher’s site not because it’s “more real” than the mommy bloggers but because it has built up more authority collectively over time. Certainly the mommy bloggers themselves are all “real” in what they write, how they put themselves out there to their communities and so on. Much depends on what they’re most concerned about.

If they had a key post that they absolutely wanted to be the primary source for, guess what? They’d better have some SEO savvy. They might need to tell the publisher to prevent the copy of their posts they provide from being spidered on the publisher’s site. Or if they allow it, then they might want to make use of the forthcoming canonical tag 2.0.

That’s not being “real.” That’s SEO. And that’s SEO that a good SEO will know — and many non-SEOs will not.

I’m glad, honestly glad, that you’re savvy enough to understand how good SEO can be incorporated into web development. I wish more web developers could do the same. But my experience has been that much good SEO gets overlooked. There are bad SEOs out there, who give the entire industry a bad name — just as there are bad bloggers, bad designers, bad cops, you name it. There are also excellent SEOs who work inside of companies as well as through agencies for hire. Don’t tarnish an entire industry that actually helps many, many people in ways I’m sure you would agree with.

For those who just can’t get enough of this subject, some of my past writings on the topic. They cover plenty of additional examples, plus the fact that Google itself recommends SEO:

• Dear Fox News: SEO Is Not Search Engine Scamming (Unless You’re Scamming Yourself): Aug. 2009
• Importance Of SEO Makes Front Page Of Los Angeles Times: March 2009
• A Bad Month For SEO’s Reputation: Feb. 2008
• From My Inbox: More Defense Of SEO: May 2007
• SEO: Real Skills That Can Protect Your Traffic: March 2007
• Why The SEO Folks Were Mad At You, Jason: Feb. 2007
• More Rounds In The “Is SEO Overrated” Debate: Jan. 2007
• Defending SEO, Yet Again!: Dec. 2006
• Yes Virginia, SEO Is Rocket Science – Defending Search Engine Optimization Once Again: Dec. 2006

See also the follow-up to this post, SEO FAQ That’s Not From The Land Of Unicorns.