User-agent: Slurp
Crawl-delay: 0.5
Disallow: .txt
Disallow: menu.html
Disallow: test.html
Disallow: .swf