Eric Enge has an outstanding interview with Matt Cutts. I have read through the entire interview and pulled out four key takeaways, but there is a lot more to be learned from the interview. I’ll list my takeaways in order of what I find to be most interesting:
(1) Matt Cutts confirmed that a 301 redirect will not always pass the full PageRank from the old URL to the new URL. Although many SEOs had a feeling that this was indeed the case, this has never been confirmed by Google until now.
(2) Google does a lot of duplicate content matching and if they see several pages on a site to be duplicate to each other, they may decide to use one URL as the main and then pass the other URL’s PageRank and value to that main URL. The only issue here is that Google is deciding for you, which is the main URL. But it is nice to see Google will pass some the PageRank of those duplicate pages to the main URL.
(3) Google crawls and indexes sites based on the PageRank of the pages on your site. Crawling might also be impacted by the number of simultaneous connections your server can handle.
(4) Blocking a URL in robots.txt will not save your “crawl budget” because they still keep the URLs in their index.