#Our spider will find any URL connected to the main body of the Web through even one link. If you #don't want your entire site to be indexed, we strongly advise that you take advantage of the Robots #Exclusion Standard by setting up a /robots.txt file. It only takes a minute, and gives you complete #control over what fraction of your site is indexed. The file looks like: User-agent: * Disallow: /MARINA Disallow: /MIMI Disallow: /SMHS2006 Disallow: /STEVEN Disallow: /TRANSFER Disallow: /VAC2001 Disallow: /VAC96 Disallow: /VAC98 Disallow: /VAC99 #Any URL matching one of these patterns will be ignored by robots visiting your site. This file is read #by Scooter every few days, so changes may not take immediate effect.