Commit graph

10 commits

Author SHA1 Message Date
1e1422e8ba Bug 7963 Parallel HTTP requests when checking URLs
Current script check-url.pl checks URL found in 856$u by sending HTTP
requests, one by one. The next request can't be sent before the previous
one get a result, which can be very slow for dead URL. I propose a new
script which send multiple requests simultaneously which improve
drastically URL checking execution time.

This script is based on AnyEvent and AnyEvent::HTTP CPAN modules.
Add new dependencies AnyEvent & AnyEvent::HTTP.

See doc: perldoc check-url-quick.pl

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
2012-10-05 11:44:52 +02:00
8caef64680 Bug 6267: [SIGNED-OFF]Fix a typo
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-06-09 14:44:17 +02:00
MJ Ray
1aef5ab44e Bug 6267 custom http user-agent in check-url.pl (fix for books.google.com 401 error)
Patch by Judit with a small change to the help wording.
Sponsored by CALYX information essentials.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
2012-04-19 12:08:17 +02:00
Andrew Elwell
9fa574f609 Bug 5385: POD Cleanups (part 1)
working through the master branch to eliminate all
podchecker warnings/errors

Actual improvement to the quality of the POD will
come later (hopefully with assistance of others)

Signed-off-by: Andrew Elwell <Andrew.Elwell@gmail.com>
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2010-11-12 10:06:55 +13:00
Lars Wirzenius
160e01daa3 Update copyright license to GPL version 2 or later for two cronjobs.
See http://lists.koha.org/pipermail/koha-devel/2010-March/011067.html
for confirmation that this is OK.

Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2010-04-07 08:43:31 -04:00
84b3c05c0d Bug 2959 - Add a timeout parameter to the URL checker
Could improve performances for DB containing URLs which can't be fetched
(good or bad) quickly.

Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2010-02-16 06:48:01 -05:00
9dee98a0c9 Bug 2959 - Cache already failed URLs to speedup check-url.pl
Based on David Schuster improvement patch.

For David:

- To send the output into an HTML file, there is no need to add a
  paramater to this script, just redirect to a file:

  check-url --html --host-prot=http://koha-pro.mylib.org \\
    > /usr/local/koha/koha-tmpl/badurls.html

- If you want as a result a table with alternate rows, use CSS and
  JavaScript. For example, with jQuery (found with google):

  <style type="text/css">
  table {width:400px; border:1px solid blue;}
  .oddrow {background-color:#E5E5E5;}
  </style>
  <script type="text/javascript"
   src="http://code.jquery.com/jquery-latest.min.js"></script>
  <script type="text/javascript">
  $(function(){
    $("table.tiger-stripe tr:even").addClass("oddrow");
  });

Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
2010-02-14 19:45:54 -05:00
Galen Charlton
b8d979abe7 minor corrections to check-url.pl
* removed ersataz YAML::XS dependency
* use 'return' instead of 'return undef'
* minor language changes

Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
2009-02-26 15:21:14 -06:00
64505d7118 URLs checker enhancement (bug #2959)
Improve URLs checker script in the way (half way) pointed out by Galen:

- A C4::URL::Checker class handle URL checking. This class is not yet
  in a separate file in C4 directory. This class would be easily
  extended to accomodate authorities URLs checking.
- Script output can now be formatted in CSV or HTML. HTML version
  link directly to MARC biblio record editor.

Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
2009-02-26 15:06:22 -06:00
37c50c6e34 A new script to check URLs in 856$u field
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
2009-02-12 15:25:14 -06:00