# Make an exception for Wget, except for the # scripted content which can't be mirrored User-agent: Wget Disallow: /cgi-bin Disallow: /cgi-local # Otherwise hide from as many bots as we can. User-agent: * Disallow: /