.Google.com's Gary Illyes shows Googlebot picks up links as opposed to "complying with" all of them, negating main paperwork.Googlebot accumulates links as opposed to following all of them in real-time.Gary Illyes discussed this info in Google's 'Look Off The Record' podcast.Google's documents negates the actual crawler habits.