Tag : spider

It would be very interesting to give statistics reports about the frequency and behaviour of spiders (search engine bots such as Googlebot, Yahoo Slurp or msnbot). How many pages do they see? How often? Are they visiting more and more pages?

Imported (and edited) from
http://www.phpmyvisites.net/bugtracker/?do=details&id=60