# Fri, 23 Feb 2007 13:33:24 -0500 # Exclude Files From All Robots: User-agent: * Disallow: /admin User-agent: msnbot Crawl-delay: 10 User-agent: Slurp Crawl-delay: 10 # End robots.txt file