Koha-community/Koha - Koha - Koha: The world's first free and open source library system

Commit Graph

Author	SHA1	Message	Date
Tomás Cohen Arazi	8ebe673931	Bug 28741: (follow-up) Silence warning Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	3 years ago
Nick Clemens	c765cfecf6	Bug 28741: Fix use of resumption token params TO test: 1 - Enable OAI-PMH server in Koha 2 - Set OAI-PMH:MaxCount to 5 3 - Define 6 sets in Admin->OAI sets configuration Can setSpec and setName to 1,2,3.... 4 - Browse to: http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListSets 5 - Click 'Show more' 6 - Keep seeing the same 5 - note the page says 'Results fetched -4-0' 7 - Apply patch 8 - Restart all the things 9 - Reload the ListSets URL above 10 - Click 'Show more' 11 - See the 6th set and note no 'Show more' 12 - Set OAI-PMH:MaxCount to 1 13 - Go to the base ListSets url again 14 - Confirm you can 'Show more' until you reach the end Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	3 years ago
Jonathan Druart	9d6d641d1f	Bug 17600: Standardize our EXPORT_OK On bug 17591 we discovered that there was something weird going on with the way we export and use subroutines/modules. This patch tries to standardize our EXPORT to use EXPORT_OK only. That way we will need to explicitely define the subroutine we want to use from a module. This patch is a squashed version of: Bug 17600: After export.pl Bug 17600: After perlimport Bug 17600: Manual changes Bug 17600: Other manual changes after second perlimports run Bug 17600: Fix tests And a lot of other manual changes. export.pl is a dirty script that can be found on bug 17600. "perlimport" is: git clone https://github.com/oalders/App-perlimports.git cd App-perlimports/ cpanm --installdeps . export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib" find . $ -name ".pl" -o -name ".pm" $ -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \; The ideas of this patch are to: * use EXPORT_OK instead of EXPORT * perltidy the EXPORT_OK list * remove '&' before the subroutine names * remove some uneeded use statements * explicitely import the subroutines we need within the controllers or modules Note that the private subroutines (starting with _) should not be exported (and not used from outside of the module except from tests). EXPORT vs EXPORT_OK (from https://www.thegeekstuff.com/2010/06/perl-exporter-examples/) """ Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members. @EXPORT and @EXPORT_OK are the two main variables used during export operation. @EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace. @EXPORT_OK does export of symbols on demand basis. """ If this patch caused a conflict with a patch you wrote prior to its push: * Make sure you are not reintroducing a "use" statement that has been removed * "$subroutine" is not exported by the C4::$MODULE module means that you need to add the subroutine to the @EXPORT_OK list * Bareword "$subroutine" not allowed while "strict subs" means that you didn't imported the subroutine from the module: - use $MODULE qw( $subroutine list ); You can also use the fully qualified namespace: C4::$MODULE::$subroutine Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	8 years ago
Ere Maijala	e74d86a343	Bug 27584: Refactor OAI-PMH paging to improve performance Includes the following optimizations: - Use next biblionumber instead of large offset in the queries. - Use unions instead of subqueries - Avoid fetching item timestamps when items are not included. Test plan: 1. Without the patch, try harvesting a Koha database with (and without for good measure) `include_items: 1` in the OAI-PMH configuration file pointed to by preference OAI-PMH:ConfFile and take note of performance. For useful metrics the database must be large enough to not fit in InnoDB buffers or OS file cache. 2. Apply the patch. 3. Run tests: prove -v t/db_dependent/OAI 4. Try again the harvesting from step 1 and compare performance with step 1. Signed-off-by: David Cook <dcook@prosentient.com.au> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	3 years ago
Jonathan Druart	d6d0116950	Bug 22824: Replace YAML::Syck with YAML::XS Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Julian Maurice <julian.maurice@biblibre.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	3 years ago
Julian Maurice	96cc447045	Bug 25898: Prohibit indirect object notation Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	4 years ago
Josef Moravec	7cd6fa6c6f	Bug 19353: Ability to use xsl template on marc21 and marcxml metadataprefixes in OAI provider Test plan: 0) apply the patchset 1) setup oai server with oaiconf file defining xsl_file for metadata format marcxml or marc21 2) play with your oai and confirm that records are transformed as expected 3) prove t/db_dependent/OAI/Server.t Signed-off-by: Martin Šťastný <martin.stastny@teamlibrary.cz> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	7 years ago
Marcel de Rooy	2bf171acaf	Bug 24052: Rename XSLT_Handler Should be XSLT::Base now. Removes old XSLT_Handler stub too (from bug 23290). Result of a git grep \| sed statement. Test plan: Run qa tools (so modules compile). Run t/db_dependent/Breeding.t Run t/db_dependent/Koha/XSLT/Base.t (This test fails when only this patch has been applied; see subsequent patch.) Enable XSLT use on results and details display. Check search results and detail view on OPAC and staff. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>	4 years ago
Marcel de Rooy	7bbb19e3bc	Bug 24809: (QA follow-up) Add prefix to prevent disambiguity This seems not to be needed. Just at the safe side. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>	4 years ago
Nick Clemens	07ff532d01	Bug 24809: Get only timestamp of deleted items attached to existing bibs when check if bib deleted To test: 1 - Edit/create a file in the koha home dir '/var/lib/koha/{instance}/OAI.yaml' with content like below: format: marcxml: metadataPrefix: marcxml metadataNamespace: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim schema: http://www.loc.gov/MARC21/slim http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd include_items: 1 2 - Set the systempreferences below to indicated values: OAI-PMH => Enable OAI-PMH:ConfFile => /var/lib/koha/{instace}/OAI.yaml 3 - View a record with items and confirm the items show: http://localhost:8080/cgi-bin/koha/oai.pl?verb=GetRecord&metadataPrefix=marcxml&identifier=KOHA-OAI-TEST:4 4 - Delete the items, then delete the record 5 - Reload the URL from step 3 6 - Internal Server Error: Empty String at /kohadevbox/koha/Koha/OAI/Server/Record.pm line 58. at /usr/share/perl/5.24/Carp.pm line 167 7 - Apply patch 8 - restart_all 9 - Repeat 10 - Reload URL from step 3 11 - Success Signed-off-by: Michal Denar <black23@gmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>	4 years ago
Ere Maijala	0a50dc1ad5	Bug 20665: Reset MySQL connection time zone in the OAI-PMH Provider TEST PLAN --------- apply unit test patch prove t/db_dependent/OAI/Server.t -- should fail apply this patch prove t/db_dependent/OAI/Server.t -- should pass run koha qa test tools Signed-off-by: Mark Tompsett <mtompset@hotmail.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	6 years ago
Julian Maurice	136c49a940	Bug 19725: OAI-PMH - Use biblio_metadata.timestamp Since bug 17196, biblioitems.timestamp is not always updated after a change in the MARC record. Filtering should be based on biblio_metadata.timestamp instead. Test plan: 1. prove t/db_dependent/OAI/Server.t 2. Verify that it SUCCEEDS Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	7 years ago
Marcel de Rooy	921e39a7ae	Bug 17807: [Follow-up] Use marcxml for marc21 prefix Not sure what happened here, but this is an easy fix. Test plan: Try oai.pl with marcxml, marc21 and oai_dc again. Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	7 years ago
Marcel de Rooy	6719eee83a	Bug 17807: Use XSLT_Handler in Koha::OAI Replacing some code in the OAI modules by a call to an existing module. Note that the xmldoc format is used in the transform call to get a xml document object. The stylesheet method of Repository now only returns the name of the xsl file to be used instead of a cached xslt object. Similar functionality inside XSLT_Handler is used when calling transform. Note: We still lack unit tests in this area. I did not see the need for adding something for stylesheet since it only returns a simple string. The other change is made in Record::new; there are no tests for this module yet and the heart of the change here is actually tested already in XSLT_Handler.t. Note: I benchmarked calls to Repository in the old and the new situation and did not see significant changes. Test plan: [1] Run t/db_dependent/OAI/Server.t [2] Run oai.pl with ListRecords and marcxml. [3] Run oai.pl with ListRecords and oai_dc. Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	8 years ago
Mark Tompsett	d5986c9b97	Bug 19040: Refactor GetMarcBiblio parameters Change parameters to a hashref. Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Looks good to me. Two calls in migration_tools/22_to_30 still in old style. Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	7 years ago
Ere Maijala	9356c050b2	Bug 15108: OAI-PMH provider improvements - Fixed date handling to use UTC as specs require. - Added support for second precision in time stamps. - Added support for marc21 metadata prefix as recommended in the guidelines (synonym for marcxml). - Improved performance of database queries especially for large collections. - Unified functionality of ListRecords and ListIdentifiers to a common base class. - If items are included in the records, their timestamps are taken into account everywhere so that whichever is the most recent (timestamp of biblioitem or any of its items) is considered the record's timestamp. - Fixed OAI.xslt to show correct record range. - Incorporated extended tests from Bug 17493 and their tweaks from Bug 15108. Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	7 years ago
Jonathan Druart	c6e488f4af	Bug 17196: Move marcxml out of the biblioitems table Two discussions on koha-devel lead to the same conclusion: biblioitems.marcxml should be moved out this table - biblio and biblioitems http://lists.koha-community.org/pipermail/koha-devel/2013-April/039239.html - biblioitems.marcxml & biblioitems.marc / HUGE performance issue ! http://lists.koha-community.org/pipermail/koha-devel/2016-July/042821.html There are several goals to do it: - Performance As Paul Poulain wrote, a simple query like SELECT publicationyear, count(publicationyear) FROM biblioitems GROUP BY publicationyear; takes more than 10min on a DB with more than 1M bibliographic records but only 3sec (!) on the same DB without the biblioitems.marcxml field Note that priori to this patch set, the biblioitems.marcxml was not retrieved systematically, but was, at least, in C4::Acquisition::GetOrdersByBiblionumber and C4::Acquisition::GetOrders - Flexibility Storing the marcxml in a specific table would allow use to store several kind of metadata (USMARC, MARCXML, MIJ, etc.) and different formats (marcflavour) - Clean code It would be a first step toward Koha::MetadataRecord for bibliographic records (not done in this patch set). Test plan: - Update the DBIC Schema - Add / Edit / Delete / Import / Export bibliographic records - Add items - Reindex records using ES - Confirm that the following scripts still work: * misc/cronjobs/delete_records_via_leader.pl * misc/migration_tools/build_oai_sets.pl - Look at the reading history at the OPAC (opac-readingrecord.pl) - At the OPAC, click on a tag, you must see the result Note: Changes in Koha/OAI/Server/ListRecords.pm is planned on bug 15108. Signed-off-by: Mason James <mtj@kohaaloha.com> Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Zeno Tajoli <z.tajoli@cineca.it> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Jonathan Druart	c69e6faa98	Bug 17778: Simplify fetch date Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Magnus Enger	ff711c48cf	Bug 17778 - Make "Earliest Registered Date" in OAI dynamic This was hardcoded to 0001-01-01. To test: - Make sure you have a couple of records, with different timestamps - Enable OAI-PMH - Check http://localhost:2201/cgi-bin/koha/oai.pl?verb=Identify and verify that "Earliest Registered Date" is 0001-01-01 - Apply the patch - Re-check the "Earliest Registered Date" and verify that it is now the lowest timestamp in your biblio table - Manipulate the timestamps and verify that "Earliest Registered Date" changes accordingly - Sign off! Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Julian Maurice	3e9dc5d51c	Bug 17744: Remove the oai_dc:dcCollection wrapper in OAI responses Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Tomás Cohen Arazi	ab531790e3	Bug 15527: Add a stylesheet for OAI-PMH to aid usability This patch introduces an XSLT file grabbed from DSpace and a couple css files. To test: - Point to your dev Koha's oai endpoint. For example: http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListMetadataFormats => FAIL: Look how ugly it is for humans. - Apply this patch - Reload => SUCCES: It is styled and provides really useful links to interact with the OAI server verbs. - Sign off :-D Sponsored-by: Universidad Empresarial Siglo 21 Signed-off-by: Frédéric Demians <f.demians@tamil.fr> Works as described. Nice looking. Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>	9 years ago
Zeno Tajoli	e884a627bb	Bug 15946 Valid namespace for OAI marcxml schema This path fixes the value of xml schema variable and the link to Library of Congress. For Dublin Core format those values are defined in UNIMARCslim2OAIDC.xsl and MARC21slim2OAIDC.xsl. For MARCXML those values are written into the code To test it: 1) Setup OAI-PMH 2) Test the site inserting the BaseURL into http://validator.oaipmh.com 3) See the problem with ListMetadataFormats [link: http://validator.oaipmh.com/#ListMetadataFormats] 4) Insert the patch 5) See the problem is over Signed-off-by: Frédéric Demians <f.demians@tamil.fr> Invalid before patch, valid after. Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com	8 years ago
Frédéric Demians	00bb0b6d57	Bug 14939: Modularize OAI Server existing classes Koha OAI server has been done in one unique .pl file because there wasn't any object model or rules in the Koha project when it has been coded. This patch modularized existing classes, putting each class in a separate file in Koha::OAI::Server namespace. UT begining. Add new dependency: Capture::Tiny Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com> OAI server moduralized succefully. Works for Debian Jessie and Wheezy. Test pass successfully Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	9 years ago

23 Commits (36ef9a219de1f76bd460c2b75cdd4bb67e79ca68)