Google’s John Mueller said on Twitter that if he had to simplify crawling priority, how Google prioritizes how it crawls the web, it would not do so by the size of a site. Instead, he would prioritizes how it crawls the web based on “the additional unique & important value provided by the pages.”
He said this on Twitter after a bunch of back and forth around questions from some SEOs on why larger sites seem to get crawled faster.
This isn’t really new but it is good to publish here.
Forum discussion at Twitter.
