Google Disables URL Removals After Bug Allows Anyone To Remove Any Site

Jul 19, 2011 at 3:10pm ET by Barry Schwartz

This morning, James Breckenridge discovered a loophole within Google’s Webmaster Tools that allowed anyone to remove any site from Google.

Both James and I sent this information to Google as soon as we heard of it. After several hours, Google has told us, “we’re still investigating this report, and to be cautious we disabled all URL removals earlier this morning.” So now, if you even own a site, you won’t be able to remove the site or pages from the site using Google’s URL removal tool.

How did this loophole work? Pretty simple as James described. You use the following URL when logged into Google Webmaster Tools:

https://www.google.com/webmasters/tools/removals-request?hl=en&siteUrl=http://{YOUR_URL}/&urlt={URL_TO_BLOCK}

Then replace {YOUR_URL} with a URL you control within Webmaster Tools, and replace {URL_TO_BLOCK} with the URL of the site you want to block.

You could block a whole site, section or single page this way, based on how you entered the URL. To block a site, use the top level domain (E.g. http://www.someurl.com/), to block a section (subfolder) use a subfolder URL (E.g. http://www.someurl.com/somefolder/) and to block a page use the specific page URL (E.g. http://www.someurl.com/somefolder/somepage.html).

I am waiting an update from Google on why this happened, if site’s were impacted and how long this was an issue.

Postscript:: Google sent us a statement that they have fixed the issue. A Google spokesperson said:

We’ve confirmed that there was an issue within the URL removal feature in our Webmaster Tools and have already pushed out a fix and re-enabled URL removals.

The URL removal feature keeps detailed records, so we’re currently reprocessing earlier removal requests to ensure their validity. Our initial examination has shown only a limited impact.

About The Author: Barry Schwartz is Search Engine Land's News Editor and owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry's personal blog is named Cartoon Barry and he can be followed on Twitter here. For more background information on Barry, see his full bio over here. See more articles by Barry Schwartz

Connect with the author via: Email | Twitter | Google+ | LinkedIn

Read before commenting! We welcome constructive comments and allow any that meet our common sense criteria. This means being respectful and polite to others. It means providing helpful information that contributes to a story or discussion. It means leaving links only that substantially add further to a discussion. Comments using foul language, being disrespectful to others or otherwise violating what we believe are common sense standards of discussion will be deleted. Comments may also be removed if they are posted from anonymous accounts. You can read more about our comments policy here.

http://twitter.com/halfbrown halfbrown

It looks like the method to do this used the Query String, so it should be easy enough for Google to (at the very least) sift through the log files to see who had been using this bug for taking out competitors. Assuming, that is, that you *had* to be logged into your GWT account as the article says.
http://www.webdesigncompany.net melvinram

@halfbrown Well I’m assuming you had to be logged in *a* GWT account, not necessarily one associated with your main sites. It’s crazy what could be done with this nuclear weapon.
http://www.esotech.org geilt

Holy crap. Imagine how long this could have been going on without being reported? Could explain or some unexpected losses in high powered industry that rely a lot on Blackhat like Insurance or Auto.

Am amazed they dont require a hash string or authentication to submit that stuff.
http://www.seroundtable.com/ Barry Schwartz

This is why I didn’t cover it here until Google removed the feature or fixed the bug.
http://www.modulussystems.com Arjun Sandhu

Barry/James – seriously? Wow! Did you try removing a site that wasn’t verified by you and did it work? I’m thinking – what if someone removed BBC or Facebook etc? That would make global news! You have no idea of the magnitude of the favour you’ve done Google. I’m looking forward to your reply – did you try removing a url?
iKiks

Any idea what happens now if a site owner really wants to remove pages, sites or sub domains. I think we need to know what steps to take.
http://www.modulussystems.com Arjun Sandhu

Just read on James’ post that the url actually did disappear off Google.
http://www.seroundtable.com/ Barry Schwartz

Arjun, I did not but James did.

ikiks, I am told Google disables the removal tool, which I wrote above. So nothing will happen, it won’t let you.
http://www.stream20.com Donal

Great spot James/Barry, I can imagine some large ethically challenged companies getting an unrelated bedroom coder to block their competition for them.
http://www.wptransfers.com Mihai Alexandru

i just tried it and it still worked for me
still showed the remove button
http://magicallinux.blogspot.com/ Andy Pieters

Well, it’s a good thing then that the RFC states that GET requests should be safe…

http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
R.M.

Well done, You have just ruined my advantage, Why spill?