# robots.txt User-agent: ia_archiver User-agent: sitecheck.internetseer.com User-agent: milou.kb.se User-agent: inktomisearch.com User-agent: Slurp User-agent: ArchitextSpider User-agent: HenryTheMiragoRobot Disallow: / User-agent: * Disallow: /js/ Disallow: /css/ Disallow: /cgi-bin/