Результаты проверки robots.txt

Здорово. А где сам robots.txt?

# $Id: robots.txt,v 1.7.2.2 2008/02/25 02:18:25 drumm Exp $

#

# robots.txt

#

# This file is to prevent the crawling and indexing of certain parts

# of your site by web crawlers and spiders run by sites like Yahoo!

# and Google. By telling these "robots" where not to go on your site,

# you save bandwidth and server resources.

#

# This file will be ignored unless it is at the root of your host:

# Used: http://example.com/robots.txt

# Ignored: http://example.com/site/robots.txt

#

# For more information about the robots.txt standard, see:

# http://www.robotstxt.org/wc/robots.html

#

# For syntax checking, see:

# http://www.sxw.org.uk/computing/robots/check.html

User-agent: *

Crawl-delay: 10

# Directories

Disallow: /database?

Disallow: /includes?

Disallow: /misc?

Disallow: /modules?

Disallow: /sites?

Disallow: /themes?

Disallow: /scripts?

Disallow: /updates?

Disallow: /profiles?

# Files

Disallow: /xmlrpc.php

Disallow: /cron.php

Disallow: /update.php

Disallow: /install.php

Disallow: /INSTALL.txt

Disallow: /INSTALL.mysql.txt

Disallow: /INSTALL.pgsql.txt

Disallow: /CHANGELOG.txt

Disallow: /MAINTAINERS.txt

Disallow: /LICENSE.txt

Disallow: /UPGRADE.txt

# Paths (clean URLs)

Disallow: /admin/

Disallow: /comment/reply?

Disallow: /contact?

Disallow: /logout?

Disallow: /node/add?

Disallow: /search?

Disallow: /user/register?

Disallow: /user/password?

Disallow: /user/login/

# Paths (no clean URLs)

Disallow: /?q=admin?

Disallow: /?q=comment/reply?

Disallow: /?q=contact?

Disallow: /?q=logout?

Disallow: /?q=node/add?

Disallow: /?q=search?

Disallow: /?q=user/password?

Disallow: /?q=user/register?

Disallow: /?q=user/login?

Ну убери комментарии)

Disallow: /comment/reply? ????оно? просто убрать?

:):):) вот это: # ......

))))))))))))))))))

А теперь правильный?:

User-agent: *

Crawl-delay: 10

Directories

Disallow: /database?

Disallow: /includes?

Disallow: /misc?

Disallow: /modules?

Disallow: /sites?

Disallow: /themes?

Disallow: /scripts?

Disallow: /updates?

Disallow: /profiles?

Files

Disallow: /xmlrpc.php

Disallow: /cron.php

Disallow: /update.php

Disallow: /install.php

Disallow: /INSTALL.mysql.txt

Disallow: /INSTALL.pgsql.txt

Disallow: /MAINTAINERS.txt

Paths (clean URLs)

Disallow: /admin/

Disallow: /comment/reply?

Disallow: /contact?

Disallow: /logout?

Disallow: /node/add?

Disallow: /search?

Disallow: /user/register?

Disallow: /user/password?

Disallow: /user/login/

Paths (no clean URLs)

Disallow: /?q=admin?

Disallow: /?q=comment/reply?

Disallow: /?q=contact?

Disallow: /?q=logout?

Disallow: /?q=node/add?

Disallow: /?q=search?

Disallow: /?q=user/password?

Disallow: /?q=user/register?

Disallow: /?q=user/login?

User-agent: *

Disallow: /database?

Disallow: /includes?

Disallow: /misc?

Disallow: /modules?

Disallow: /sites?

Disallow: /themes?

Disallow: /scripts?

Disallow: /updates?

Disallow: /profiles?

Disallow: /xmlrpc.php

Disallow: /cron.php

Disallow: /update.php

Disallow: /install.php

Disallow: /INSTALL.mysql.txt

Disallow: /INSTALL.pgsql.txt

Disallow: /MAINTAINERS.txt

Disallow: /admin/

Disallow: /comment/reply?

Disallow: /contact?

Disallow: /logout?

Disallow: /node/add?

Disallow: /search?

Disallow: /user/register?

Disallow: /user/password?

Disallow: /user/login/

Disallow: /?q=admin?

Disallow: /?q=comment/reply?

Disallow: /?q=contact?

Disallow: /?q=logout?

Disallow: /?q=node/add?

Disallow: /?q=search?

Disallow: /?q=user/password?

Disallow: /?q=user/register?

Disallow: /?q=user/login?

СПАСИБО!!!