The BYZ Bloggle
Check this page regularly for news of software updates, new releases, important announcements and other things we think might be of interest.Please feel free to leave comments. We appreciate and value your feedback whether good or bad.
Yandex
Fri, Dec 2009 06:55
| yandex, robots.txt, serach engines
Yandex is an over zealous Russian search engine that will scrupulously index as many pages as your site can offer up and it’s spider enjoys it’s work. Yandex may be industrious but unless you have pages of interest to Russian speakers you may want to restrict it’s spidering of your site because it can use a lot of your valuable bandwidth allowance.
To prevent Yandex from crawling simply add the following to your robots.txt file:
User-agent: yandex bot
Disallow: /
To prevent Yandex from crawling simply add the following to your robots.txt file:
User-agent: yandex bot
Disallow: /
Comments
