Tutti gli Spider di Google

giorgiotave

Tutti gli Spider di Google

[TABLE="class: grid, width: 900"]

[TH="bgcolor: #6199DF, align: left"]Crawler[/TH]
[TH="bgcolor:, align: left"]User-agents (+ less-specific alternatives) followed in robots.txt, robots meta tags & X-Robots-Tag[/TH]
[TH="bgcolor:, align: left"]User-agent in HTTP(S) requests[/TH]
[TH="bgcolor:, align: left"]Comments + Documentation URL[/TH]
[/TR]

[TD="align: left"]Googlebot (web)[/TD]
[TD="align: left"]Googlebot[/TD]
[TD="align: left"]Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Alternate (rarely used):
Googlebot/2.1 (+http://www.google.com/bot.html)[/TD]
[TD="align: left"]Generic Googlebot crawler for web-search
http://www.google.com/bot.html[/TD]
[/TR]

[TD="align: left"]Googlebot News[/TD]
[TD="align: left"]Googlebot-News
(Googlebot)[/TD]
[TD="align: left"]Googlebot-News[/TD]
[TD="align: left"]For Google News[/TD]
[/TR]

[TD="align: left"]Googlebot Images[/TD]
[TD="align: left"]Googlebot-Image
(Googlebot)[/TD]
[TD="align: left"]Googlebot-Image/1.0[/TD]
[TD="align: left"]For Image Search[/TD]
[/TR]

[TD="align: left"]Googlebot Video[/TD]
[TD="align: left"]Googlebot-Video
(Googlebot)[/TD]
[TD="align: left"]Googlebot-Video/1.0[/TD]
[TD="align: left"]For Video Search[/TD]
[/TR]

[TD="align: left"]Googlebot Mobile[/TD]
[TD="align: left"]Googlebot-Mobile
(Googlebot)[/TD]
[TD="align: left"][various mobile device types] (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)[/TD]
[TD="align: left"]For Google Mobile web-search results
http://www.google.com/support/mobile/bin/answer.py?answer=37425[/TD]
[/TR]

[TD="align: left"]Google Mobile AdSense[/TD]
[TD="align: left"]Mediapartners-Google
Mediapartners
(Googlebot)[/TD]
[TD="align: left"][various mobile device types] (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)[/TD]
[TD="align: left"][/TD]
[/TR]

[TD="align: left"]Google AdSense[/TD]
[TD="align: left"]Mediapartners-Google
Mediapartners
(Googlebot)[/TD]
[TD="align: left"]Mediapartners-Google[/TD]
[TD="align: left"][/TD]
[/TR]

[TD="align: left"]Google AdsBot landing page quality check[/TD]
[TD="align: left"]AdsBot-Google
(Given the special nature of this crawler, only directives for this user-agent are followed.)[/TD]
[TD="align: left"]AdsBot-Google (+http://www.google.com/adsbot.html)[/TD]
[TD="align: left"]Only visits landing pages used in AdWords campaigns. See http://www.google.com/adsbot.html[/TD]
[/TR]
[/TABLE]

Inoltre Google ci fa sapere che:

Googlebot-Mobile User-Agents

Attualmente in uso:

Feature phones Googlebot-Mobile:
SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
Smartphone Googlebot-Mobile:
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)