Koha-community/Koha
15
7
Fork 0
Code Issues Releases Wiki Activity

Bug 6411 add another example to README.robots

Browse source 
Signed-off-by: Magnus Enger <magnus@enger.priv.no>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
This commit is contained in:
JAMES Mason 2011-05-30 20:54:24 +12:00 committed by Chris Cormack
parent e653e8ce0b
commit 2e8fe364cb
1 changed files with 10 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
README.robots
View file

@ -5,13 +5,23 @@ look for the file /robots.txt. If this file is found and has lines that apply
to them they will do as instructed. A very basic robots.txt follow as an
example:
-------------------------------------------
# go away
User-agent: *
Disallow: /
-------------------------------------------
This tells every search engine that cares (User-agent: *) to not index the site
(Disallow everything past /).
Another slightly more intelligent robots.txt file example allows for some bot indexing (good for your site in google, etc), but also stops your Koha from getting thrashing by ignoring URLs that cause heavy search load
-------------------------------------------
# do some indexing, but dont index search URLs
User-agent: *
Disallow: /cgi-bin/koha/opac-search.pl
-------------------------------------------
If you have installed Koha to /usr/local/koha3 then this file would be placed
in the directory /usr/local/koha3/opac/htdocs/. This should prevent search
engines from browsing every biblio record, and every view of each record, on