From 62f43c04e934d34451d6aaadf7e72fa3ada04fcb Mon Sep 17 00:00:00 2001 From: Michael Hafen Date: Fri, 19 Feb 2010 09:34:10 -0700 Subject: [PATCH] Add instructions to INSTALL and README.robots about robots.txt Instructions are given in INSTALL and README.robots about adding a robots.txt file to the opac to prevent search engines from indexing Koha. Signed-off-by: Galen Charlton --- INSTALL | 1 + README.robots | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 README.robots diff --git a/INSTALL b/INSTALL index df0990c85e..4e8c20bbd5 100644 --- a/INSTALL +++ b/INSTALL @@ -56,6 +56,7 @@ Default installation instructions: (note that you will want to run Zebra in daemon mode for a production system) 9. Browse to http://servername:8080/ and answer the questions +10. Optionally add a robots.txt file. See README.robots for details OR if you want to install all dependencies from CPAN and are root, you can replace steps 1-3 with "perl install-CPAN.pl" but this is non-standard and diff --git a/README.robots b/README.robots new file mode 100644 index 0000000000..a08371dd07 --- /dev/null +++ b/README.robots @@ -0,0 +1,18 @@ +The robots.txt file. + +Search engines, when looking for sites to show in search results, will first +look for the file /robots.txt. If this file is found and has lines that apply +to them they will do as instructed. A very basic robots.txt follow as an +example: + +# go away +User-agent: * +Disallow: / + +This tells every search engine that cares (User-agent: *) to not index the site +(Disallow everything past /). + +If you have installed Koha to /usr/local/koha3 then this file would be placed +in the directory /usr/local/koha3/opac/htdocs/. This should prevent search +engines from browsing every biblio record, and every view of each record, on +your Koha install periodically.