Adding robots.txt

[Christopher Greiner]

Hi Sympa users,

I'm looking to add a robots.txt file as it's been brought to our
attention a couple of our pages have been indexed in Google with some
user email addresses being displayed.

I've since added the spam_protection directive in sympa.conf and
requested removal of the offending pages via Google's webmaster tools.
I'd like to add a robots.txt file to avoid our sympa subdomain getting
indexed in the future. I've had a search and can't find any information
on how to do this. Any pointers?

Thanks for your help.

Regards,

Chris Greiner
--
Christopher Greiner
Université de Lausanne
Centre informatique
Amphimax
CH-1015 Lausanne

E-mail: Christopher.Greiner [at] unil.ch
Tel: [phone]
URL: http://www.unil.ch/ci/

[Miles Fidelman]

Post by Christopher Greiner
Hi Sympa users,
I'm looking to add a robots.txt file as it's been brought to our
attention a couple of our pages have been indexed in Google with some
user email addresses being displayed.
I've since added the spam_protection directive in sympa.conf and
requested removal of the offending pages via Google's webmaster tools.
I'd like to add a robots.txt file to avoid our sympa subdomain getting
indexed in the future. I've had a search and can't find any information
on how to do this. Any pointers?

Simplest thing would be to add a static page to your web server - don't
need to involve sympa at all to do this.

So.. if your server is lists.unil.ch, then create a text page at
lists.unil.ch/robots.txt - containing something like:

User-agent: *
Disallow: *

If you want to be a bit more granular, and allow indexing of some things, see the description of robots.txt at
http://www.robotstxt.org/orig.html

Miles Fidelman