Topic: Web scraping

The Definitive Guide To Technical Mobile SEO

At SMX Advanced, I moderated a panel about technical SEO. Google’s Maile Ohye spoke about SEO best practices for technical implementation of mobile sites based on how Google crawls, indexed, and ranks mobile content and presents it to searchers on mobile devices. She also talked about Google’s recent announcement that the mobile user experience is […]

Yes, Google Says It’s Having Problems Listing iTunes Preview Pages

Yes, it is harder to find pages to iTunes apps in Google. But no, the company says, it’s not part of some nefarious plot. Rather, Google’s having technical problems gathering iTunes Preview pages, an issue it’s working to solve. Google has confirmed the issue, saying: We’ve been having some issues fetching pages from the iTunes […]

Hijacking Google Search Results With Duplicate Content

Dan Petrovic has explained how he hijacked a few pages in Google to show his copied version over the original version of the page. For example, he was able to confuse Google into thinking a page on MarketBizz should really show on dejanseo.com.au instead of on marketbizz.nl. How did he do it? He simply copied […]

New Alerts For Crawl Errors Via Google Webmaster Tools

Google announced they are now sending out Google Webmaster Tools alerts for crawl errors. Google grouped the errors in two buckets. (1) Site Error alerts for major site-wide problems (2) URL Error anomaly alerts for potentially less critical issues The Site-Wide Problems Alerts Can Include: Your DNS server is down or misconfigured. Your web server […]

SMX Advanced 2012: Pagination & Canonicalization For The Pros

Pagination & Canonicalization For The Pros Presentations from SMX Advanced, June 2012 Using the pagination tag with optional parameters, sort orders, and filters. Are there still reasons to use robots.txt or noindex? Can a canonical tag really replace a 301? How do you keep your IIS=based site from infinite redirect loops when you canonicalize default […]

Google Webmaster Tools Revamps Crawl Errors, But Is It For The Better?

Google has just revamped the crawl errors data available in webmaster tools. Crawl errors are issues Googlebot encountered while crawling your site, so useful stuff! I originally started this article by writing that in most cases, these changes are for the better and in only a few (really maddening) cases, useful functionality has been removed. […]

WSJ Pulls Back On What Google Searchers Can Read For Free

Are you used to using Google as a way around the Wall Street Journal’s paywall? Think again. The WSJ has been holding back stories available through Google’s “First Click Free” program, a move that I suspect other newspapers might soon emulate. Over the past few weeks, I’ve noticed that I wasn’t able to read some […]

“Scroogle.org Is Gone Forever” Says Site Owner

A week ago, we reported that Scroogle was being blocked by Google and searches on the privacy search engine were no longer working. Since then, the site continued not to work and now it is completely offline. The owner, Daniel Brandt, said he took it down “forever.” He blames both Google and DDOS attack by […]

Scroogle Says Google Is Blocking Their Privacy Search Engine

If you try to conduct a search at Scroogle, the portal that lets you search Google in private, Scroogle will tell you that Google is blocking them. I just tested it and Scroogle returns an error message that reads, “so sorry… Google is temporarily blocking this Scroogle server.” This happened back in May 2010 as […]

When Is the Super Bowl Start Time? The NFL Finally Gets It Right

Super Bowl 46 kicks off on February 5, 2012 at 6:30pm EST on NBC. Amazingly enough, I found this information by searching on Google and clicking on the second result: nfl.com. Amazing because every year, football fans flock to search engines searching for the start time, and until now, organizations like the NFL, the playing […]

SMX East 2011: Duplication, Aggregation, Syndication, Affiliates, Scraping, and Information Architecture

Duplication, Aggregation, Syndication, Affiliates, Scraping, and Information Architecture Presentations from SMX East – NYC, NY, September 2011 We’ll sort through the bewildering world of near duplication due to issues such as manufacturer feeds into your site, syndication to other sites, international sites, and aggregation of data sources. How can a well-planned information architecture mimimize the […]

Live Blog: Yelp, NexTag & Others At The US Senate Hearing

OK, it’s round two. And I can tell you no, don’t expect much. How do you clear the press table when Google competitors are about to testify at a Senate hearing? Have Eric Schmidt do a post-game recap in the hearing room hallway. He’s done, I’m back, will catch up as best I can. NOTE: […]

How Twitter’s Technical Infrastructure Issues Are Impacting Google Search Results

In the last few days, several noticed that the Google Toolbar PageRank value for www.twitter.com had plummeted to 0 (it’s now back to its previous 9). Was Google punishing Twitter for how things went down with the feed expiration that temporarily derailed Google’s Realtime search efforts? Nah, Twitter’s website has just been suffering from technical […]

Google Realtime Search & The Aftermath Of The Google-Twitter Split

Last Friday, Twitter quietly shutdown its “firehose” of tweet data that was being piped to Google. Like a gas station no longer getting deliveries, Google in turn effectively had to hang a “Closed” sign on its Google Realtime Search service. What happened, and what’s next for those who depended on Google to get some of […]

Coming Soon: Google Panda Update 2.2

One of the bigger pieces of news — certainly for webmasters and SEOs — from our SMX Advanced conference this week is that Google will soon be releasing version 2.2 of the Panda algorithm update. Google’s Matt Cutts confirmed that during Tuesday’s keynote question-and-answer session with Danny Sullivan. Cutts said that the update has been […]

SearchCap: The Day In Search, February 3, 2011

Below is what happened in search today, as reported on Search Engine Land and from other places across the web. From Search Engine Land: Did A Google Employee Just Register Hiybbprqag.com To Help Recruit New Employees? That’s a screenshot of Google’s “Jobs” page, which can be found at https://www.google.com/intl/en/jobs/index.html. But it can also be reached […]

Review Sites’ Rancor Rises With Prominence of Google Place Pages

Friends or enemies? Partners or competitors? Google’s relationship with review sites like TripAdvisor and Yelp is as complicated as ever, and the disputes are likely to increase as the search giant pursues mobile and local ambitions that center around its controversial Place pages. The rancor stems from the way Google aggregates content from various review […]

SEO

Google launches algorithm to fight low-quality scraper content

Earlier this week, Google’s Matt Cutts announced the company would be releasing a new on-page spam detection technique to reduce the amount of content spam we see in Google. Cutts just confirmed the rumors that this algorithm is indeed live. Matt said the change to their algorithm to prevent low-quality scraper content in Google’s index […]

TripAdvisor Back From Its Google Vacation?

Last week it appeared that hotels reviews site TripAdvisor was blocking Google from showing its content on Places Pages. TripAdvisor never confirmed that it was doing so and Google attributed the absence of TripAdvisor reviews on Places to a technical problem. However what you might call “circumstance evidence,” as well as comments made by TripAdvisor […]

TripAdvisor Blocks Google: The Start Of A Larger Trend?

First there was the Google-Yelp dispute over reviews in Places (which was resolved). Now TripAdvisor appears to be preventing Google from showing its reviews on Place Pages. The first to report this yesterday was Tnooz. According to the article: Google is no longer able to stream in reviews from TripAdvisor to Places pages after the […]

Google Creates “Source” Meta Tags To Help ID Original News Sources

With the amount of content published online seeming to expand dramatically every year, Google says it’s experimenting with two new meta tags that it hopes will help it identify the original sources of online content. They’re called syndication-source and original-source and here’s a look at what they do and how publishers can use them. What […]

Swingly: New Search Engine Reveals Just The Facts

The days of search engines merely being a middle man are over. Search isn’t about sending users to the right web page, it’s about getting them the information they want as soon as possible — sometimes it’s a web page, but it might also be a sports score, movie showtimes, election information, or some other […]

Did Google Screw Scroogle?

Scroogle, a site designed for those who don’t want Google tracking their searches back to them, had some hiccups since the new Google design went live on May 5th. Scroogle worked by “scraping” the Google search results from https://www.google.com/ie, a service for Internet Explorer 6 users. Google has decided to discontinue that service redirect it […]