User-Agent: * #the * specifies all agents or robots Disallow: /extranet/ Disallow: /eextranet/ Disallow: /vpn/ Disallow: /downloads/ Disallow: /apps/ #the / indicates all documents under the server root