Doug Smythies - Google WebMaster tools complain about 404 return codes from GoogleBot invented file names. 2011.04.12

Summary:
Sometimes GoogleBot will invent an html file name, based on random characters, for a known to exist sub-folder.
GoogleBot does this specifically to check what the web server responds with for a 404 (not found) error. Google WebMaster tools recommend that site owners make some friendly and polite 404 response page.
Myself, I have never understood why GoogleBot does this check so often, nor why it will do it day after day on the same sub-folder, nor why I should bother to make my own 404 response web page.
In my opinion, the user will get a more consistant look and feel from allowing their browser to supply the 404 information.

However, it is totally outrageous that the Google WebMaster tools will list the 404 (not found) return codes from the random GoogleBot invented URL's as crawl errors. For goodness sakes, how utterly ridiculous.

Example:
First, some GoogleBot invented URL entries in my apache web server access log file:

access.log.1:66.249.68.53 - - [28/Mar/2011:11:47:03 -0700] "GET /lacrosse/Junior_B_Adanacs_2008/Game4Web/pages/mcfxxvugcxivu.html HTTP/1.1" 404 547 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.2:66.249.71.117 - - [26/Mar/2011:14:02:23 -0700] "GET /lacrosse/Junior_B_Adanacs_2008/Game4Web/pages/lrsrbsvgq.html HTTP/1.1" 404 544 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.2:66.249.72.82 - - [25/Mar/2011:13:07:45 -0700] "GET /lacrosse/Junior_B_Adanacs_2008/Game4Web/pages/dyzixnktfrld.html HTTP/1.1" 404 547 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.2:66.249.68.53 - - [23/Mar/2011:14:22:32 -0700] "GET /lacrosse/Junior_B_Adanacs_2008/Game4Web/pages/chlseklgkefq.html HTTP/1.1" 404 546 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.2:66.249.68.53 - - [22/Mar/2011:15:11:23 -0700] "GET /lacrosse/Junior_B_Adanacs_2008/Game4Web/pages/nrifszmhusc.html HTTP/1.1" 404 545 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.4:66.249.68.53 - - [11/Mar/2011:16:49:49 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/mfhbcrir.html HTTP/1.1" 404 550 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.4:66.249.68.53 - - [10/Mar/2011:14:40:37 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/hxptvkjfuber.html HTTP/1.1" 404 553 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.4:66.249.68.53 - - [09/Mar/2011:14:04:36 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/mwgfcdovryfyy.html HTTP/1.1" 404 555 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.4:66.249.68.53 - - [08/Mar/2011:13:46:17 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/luowfqskerhtw.html HTTP/1.1" 404 554 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.4:66.249.68.53 - - [07/Mar/2011:14:22:32 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/jkkrbflfwjvqu.html HTTP/1.1" 404 556 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.6:66.249.68.53 - - [24/Feb/2011:16:32:54 -0800] "GET /lacrosse/Intermediate_B_Adanacs/0806/pages/uqrdptxkzadqcf.html HTTP/1.1" 404 544 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.9:66.249.68.33 - - [04/Feb/2011:01:02:41 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/fykbjxmwrxi.html HTTP/1.1" 404 550 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.9:66.249.68.33 - - [03/Feb/2011:02:56:25 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/snlskebbefbyzlhc.html HTTP/1.1" 404 553 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.9:66.249.68.33 - - [01/Feb/2011:02:22:27 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/pekwffybwz.html HTTP/1.1" 404 551 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.9:66.249.68.33 - - [31/Jan/2011:03:18:16 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/vbubhngdc.html HTTP/1.1" 404 549 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.10:66.249.68.33 - - [30/Jan/2011:02:54:36 -0800] "GET /lacrosse/Junior_B_Adanacs_2008/Gold_Metal_Game_Web/pages/dvibjysoc.html HTTP/1.1" 404 549 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.11:66.249.68.33 - - [19/Jan/2011:06:14:16 -0800] "GET /lacrosse/Intermediate_B_Adanacs/0616/pages/szxwfkcymu.html HTTP/1.1" 404 539 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.11:66.249.68.33 - - [18/Jan/2011:07:17:17 -0800] "GET /lacrosse/Intermediate_B_Adanacs/0616/pages/cvvvpsxtutydzbm.html HTTP/1.1" 404 542 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.11:66.249.68.33 - - [17/Jan/2011:07:58:16 -0800] "GET /lacrosse/Intermediate_B_Adanacs/0616/pages/mhsitqehdy.html HTTP/1.1" 404 536 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.11:66.249.68.33 - - [16/Jan/2011:08:23:32 -0800] "GET /lacrosse/Intermediate_B_Adanacs/0616/pages/zmnxpojdmvsrdc.html HTTP/1.1" 404 542 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.12:66.249.68.33 - - [15/Jan/2011:08:46:40 -0800] "GET /lacrosse/Intermediate_B_Adanacs/0616/pages/euholrfgxat.html HTTP/1.1" 404 538 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
access.log.13:66.249.68.33 - - [04/Jan/2011:05:03:57 -0800] "GET /lacrosse/Intermediate_B_Adanacs/0806/pages/aodwbzia.html HTTP/1.1" 404 537 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Second, the eventual crawl errors listed on Google WebMaster tools:


Of course "Linked From" is unavailable, because GoogleBot invented the URL in the first place.