Koha-community/Koha - Koha - Koha: The world's first free and open source library system

Commit Graph

Author	SHA1	Message	Date
Jonathan Druart	9d6d641d1f	Bug 17600: Standardize our EXPORT_OK On bug 17591 we discovered that there was something weird going on with the way we export and use subroutines/modules. This patch tries to standardize our EXPORT to use EXPORT_OK only. That way we will need to explicitely define the subroutine we want to use from a module. This patch is a squashed version of: Bug 17600: After export.pl Bug 17600: After perlimport Bug 17600: Manual changes Bug 17600: Other manual changes after second perlimports run Bug 17600: Fix tests And a lot of other manual changes. export.pl is a dirty script that can be found on bug 17600. "perlimport" is: git clone https://github.com/oalders/App-perlimports.git cd App-perlimports/ cpanm --installdeps . export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib" find . $ -name ".pl" -o -name ".pm" $ -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \; The ideas of this patch are to: * use EXPORT_OK instead of EXPORT * perltidy the EXPORT_OK list * remove '&' before the subroutine names * remove some uneeded use statements * explicitely import the subroutines we need within the controllers or modules Note that the private subroutines (starting with _) should not be exported (and not used from outside of the module except from tests). EXPORT vs EXPORT_OK (from https://www.thegeekstuff.com/2010/06/perl-exporter-examples/) """ Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members. @EXPORT and @EXPORT_OK are the two main variables used during export operation. @EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace. @EXPORT_OK does export of symbols on demand basis. """ If this patch caused a conflict with a patch you wrote prior to its push: * Make sure you are not reintroducing a "use" statement that has been removed * "$subroutine" is not exported by the C4::$MODULE module means that you need to add the subroutine to the @EXPORT_OK list * Bareword "$subroutine" not allowed while "strict subs" means that you didn't imported the subroutine from the module: - use $MODULE qw( $subroutine list ); You can also use the fully qualified namespace: C4::$MODULE::$subroutine Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	8 years ago
Tomás Cohen Arazi	58c275e04b	Bug 23583: Handle OpacHiddenItems with yaml_preference This patch makes all places in Koha that rely on OpacHiddenItems actually use C4::Context->yaml_preference instead of manually calling the YAML libraries and handling it. To test: 1. Apply this patch 2. Run: $ kshell k$ prove t/db_dependent/Items* \ t/db_dependent/Koha/Item* \ t/db_dependent/Koha/Filter/EmbedItems.t => SUCCESS: Tests pass! 3. Try hiding things with opac-ISBDdetail.pl and opac-search.pl => SUCCESS: Things work the same! 4. Sign off :-D Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	3 years ago
James O'Keeffe	2b607d8414	Bug 24000: Some modules do not return 1 This patch adds "1;" to the end of the following perl files: Koha/Filter/MARC/EmbedItemsAvailability.pm Koha/Filter/MARC/EmbedSeeFromHeadings.pm Koha/Filter/MARC/Null.pm Koha/Item/Search/Field.pm Koha/SearchEngine.pm misc/translator/VerboseWarnings.pm t/lib/Koha/Plugin/Test.pm This indicates the succesful execution of the initialization code. Test plan: Ensure that there are no other perl files that need "1;", but dont have it. Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	3 years ago
Jonathan Druart	fafcbff015	Bug 27673: Fix encoding issues There is a difference between YAML::Load and YAML::XS::Load From YAML::XS pod: "YAML::XS only deals with streams of utf8 octets" Test plan: We are going to test 1 occurence and QA will confirm others don't contain typos. 0. Don't apply the patches 1. Create a new itemtype with code=❤️ 2. Create a new item using this itemtype (to biblionumber=1 will work) 3. Fill OpacHiddenItems with itype: [❤️] 4. Search for "street shuffle" or any terms that will return the biblio Notice that the item is there (there is an error in logs) 5. Apply the patches 6. Repeat 4 and confirm that the item is now hidden Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	3 years ago
Jonathan Druart	46f7239b08	Bug 27673: Replace YAML with YAML::XS From tht YAML pod: """ This module has been released to CPAN as YAML::Old, and soon YAML.pm will be changed to just be a frontend interface module for all the various Perl YAML implementation modules, including YAML::Old. If you want robust and fast YAML processing using the normal Dump/Load API, please consider switching to YAML::XS. It is by far the best Perl module for YAML at this time. It requires that you have a C compiler, since it is written in C. """ See also https://gitlab.com/koha-community/qa-test-tools/-/merge_requests/35 Test plan: Try some place where YAML::XS is not used and confirm that it works correctly QA note: This patch removes some uses of YAML that were not useful Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	3 years ago
Katrin Fischer	ed066caf58	Bug 26015: Terminology: Fix tons more staff clients to staff interfaces This takes care of more occurences of staff client and changes it to staff interface, including in code comments. To test: - I think in this case careful code review is what we look for. Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>	4 years ago
Tomás Cohen Arazi	df58c3e2c9	Bug 25019: Make the ViewPolicy filter initialization standard This patch makes the ViewPolicy filter use the 'params' accessor instead of relying of ->{options} which has no accessor. This will allow interacting with the filter object be similar through all the filters in the chain. To test, we just need to verify no behaviour change takes place: 1. Run: $ kshell k$ prove t/db_dependent/Filter_MARC_ViewPolicy.t => SUCCESS: Tests pass 2. Apply this patch 3. Repeat 1 => SUCCESS: Tests still pass! 4. Sign off :-D Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>	4 years ago
Tomás Cohen Arazi	6189f25dfa	Bug 23793: Add an EmbedItems RecordProcessor filter for MARC::Record objects This patch introduces a RecordProcessor filter for MARC::Record objects that embeds MARC::Field's for each item that is passed to the processor. Target usage: my $biblio = Koha::Biblios->find( $biblio_id, { prefetch => [ items, metadata ] } ); my $opachiddenitems_rules; eval { my $yaml = C4::Context->preference('OpacHiddenItems') . "\n\n"; $opachiddenitems_rules = YAML::Load($yaml); }; my @items = grep { !$_->hidden_in_opac({ rules => $opachiddenitems_rules }) @{$biblio->items->as_list}; my $record = $biblio->metadata->record; my $processor = Koha::RecordProcessor->new( { filters => ('EmbedItems'), options => { items => \@items } } ); $processor->process( $record ); Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>	5 years ago
Josef Moravec	68eeefa07e	Bug 22721: Remove frameworkcode parameter in GetMarcFromKohaField calls Test plan: Run tests, at least: t/db_dependent/Biblio.t t/db_dependent/Biblio/TransformHtmlToMarc.t t/db_dependent/Charset.t t/db_dependent/Circulation/GetTopIssues.t t/db_dependent/Filter_MARC_ViewPolicy.t t/db_dependent/ImportBatch.t t/db_dependent/Items.t t/db_dependent/Items/AutomaticItemModificationByAge.t t/db_dependent/Items/GetItemsForInventory.t t/db_dependent/Koha/Filter/EmbedItemsAvailability.t t/db_dependent/Serials.t t/db_dependent/XISBN.t t/db_dependent/FrameworkPlugin.t Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Michal Denar <black23@gmail.com> Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>	5 years ago
Julian Maurice	61ca4deee3	Bug 18579: Fix call to append_fields, fix tests MARC::Record::append_fields takes a list of MARC::Field (not an arrayref) Use $record->subfield() instead of $record->field()->subfield() to avoid errors when field doesn't exist Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	7 years ago
Tomás Cohen Arazi	542ac21b94	Bug 18579: Make EmbedItemsAvailability handle control fields correctly This patch makes the EmbedItemsAvailability filter handle the biblio.biblionumber > control field mapping gracefully. Right now, it was assuming a regular data field was used, and such is not the case for UNIMARC (001). To test: - Apply the tests patch - Run: $ sudo koha-shell kohadev k$ cd kohaclone k$ prove t/db_dependent/Koha/Filter/EmbedItemsAvailability.t => FAIL: Tests fail to run - Apply this patch - Run: k$ prove t/db_dependent/Koha/Filter/EmbedItemsAvailability.t => SUCCESS: Tests pass! - Sign off :-D Sponsored-by: ByWater Solutions Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	7 years ago
Marcel de Rooy	b038bb1310	Bug 18208: [QA Follow-up] Get count from Koha::Items Removes the loop by moving the onloan test to the query. Test plan: Run the test again. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	7 years ago
Tomás Cohen Arazi	b27b77307c	Bug 18208: Add RecordProcessor filter to inject not onloan count to MARC records This patch adds a new filter for MARC records to be used with Koha::RecordProcessor. It's purpose is to inject the information about items not-onloan in a fixed subfield, 999$x. To test: - Apply the patch - Run: $ prove t/db_dependent/Koha/Filter/EmbedItemsAvailability.t => SUCCESS: Tests pass! - Sign off :-D Sponsored-by: ByWater Solutions Followed test plan, test passes OK Signed-off-by: Marc Véron <veron@veron.ch> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	7 years ago
Jacek Ablewicz	7bdc106c98	Bug 16365: Selectively introduce GetMarcStructure() "unsafe" variant for better performance GetMarcStructure() currently uses Koha::Cache in the "safe" mode (returning deep copy of the result data structure by default), which causes numerous performance issues in many Koha scripts. Switching it to the "unsafe" mode globally (2nd patch from Bug 16140) resolves those issues, but ensuring that it is regression-free (and that it will stay that way in the future) is far from easy. This patch proposes a bit more manageable solution, it introduces a possibility to use "unsafe" variant selectively (only in those places in the code where GetMarcStructure() is called repetitively). That way, amount of the code that needs to be audited for possible problems gets vastly reduced, without any performance trade-offs. Test plan: 1) Have a look at the code and audit the parts affected by this patch for possible regressions Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Amended the POD of GetMarcStructure, removing a TODO. NOTE: GetAuthorisedValueDesc, as called in C4::XSLT::transformMARCXML4XSLT and by GetISBDView, GetMarcAuthors in C4::Biblio, may autovivify some hash entries in tagslib. Same for Koha/Filter/MARC/ViewPolicy.pm, sub filter. No reason however to worry; our use of this structure in Koha does not depend on the existence of intermediate hash keys. (We seem to be safe as long as $tagslib->{$tag}->{$subfield}->{tab} and/or hidden are not filled.) Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Mark Tompsett	dd8352bb8b	Bug 11592: Add should_hide_marc method to filter Add should_hide_marc to ViewPolicy filter Add should_hide_marc tests to t/db_dependent/Filter_MARC_ViewPolicy.t Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>	8 years ago
Mark Tompsett	4e1289a55d	Bug 15870: potential follow up to comment #8 This patch: - makes the Koha::RecordProcessor code more clear by removing the unnecessary newrecord variable. - revises the filter to be more clear about the expectation that operations are done directly on the record parameter. TEST PLAN --------- prove t/RecordProcessor.t prove t/db_dependent/Filter_MARC_ViewPolicy.t run koha qa test tools Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com> Works as advertised. NO koha-qa errors Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>	8 years ago
Mark Tompsett	c9170233c9	Bug 15870: Follow-up of filter and tests This patch: - improves perlcritic messages in the filter and tests. - changes should display logic to should hide logic to simplify filter. - perltidies the scripts - debugs the issues outstanding on the comprehensive tests provided in the second commit. Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>	8 years ago
Mark Tompsett	25ffd8b215	Bug 15870: MARC Filter to exclude fields/subfields lacking visibility TEST PLAN --------- 0) Apply 15777 and 15871 as required. 1) Apply patch 2) prove -v t/db_dependpent/RecordProcessor_ViewPolicy.t -- all should pass. 3) koha qa test tools. BONUS 4) perlcritic -1 t/db_dependpent/RecordProcessor_ViewPolicy.t -- using the koha qa test tools perlcriticrc, it should pass though I am excluding checking print return values. (see bug 15646) Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>	8 years ago
Jonathan Druart	017699c345	Bug 16011: $VERSION - Remove the $VERSION init Mainly a perl -p -i -e 's/^.3.07.00.049.\n//' */.pm Then some adjustements Signed-off-by: Josef Moravec <josef.moravec@gmail.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar> Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>	8 years ago
Jonathan Druart	03c89fa02e	Bug 15380: Move Koha::Authority to Koha::MetadataRecord::Authority The existing Koha::Authority does not use Koha::Objects and does not exclusively deal with DB objects. It makes much sense to move it to let the space free for a Koha::Authority and Koha::Authorities modules based on Koha::Object. Signed-off-by: Marc Véron <veron@veron.ch> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com	9 years ago
Jacek Ablewicz	f3a8b7a0e1	Bug 14456: EmbedSeeFromHeadings record filter shouldn't process MARC holding fields If the system preference IncludeSeeFromInSearches is enabled, records exported for zebra indexing are being additionally processed by EmbedSeeFromHeadings record filter (right now used only in rebuild_zebra.pl script). This filter embeds 'see from' fields (extracted from authority records linked with the given biblio via $9 subfields) into target MARC record, which is then subsequently indexed in zebra. Currently all fields containing $9 are getting the same exact treatment by this filter. But on the export stage when the filter is applied, MARC record being processed already does have holdings data fields added in the previous stage (usually 952 / 995, depending on the MARC format). Problem is that holdings data fields use to have $9 subfields in them as well (mapped to item.itemnumber by default). As a consequence, some (great many in the typical setup) records exported for zebra indexing may have surplus "see from" fields added erroneously in semi-random fashion, so biblio searches would often return some completely unexpected additional results. EmbedSeeFromHeadings record filter should not process holdings fields when dealing with MARC records intended for zebra indexing. To reproduce: 1) database with as many sample or real-world biblio, item and authority records as possible is recommended for testing purposes 2) enable IncludeSeeFromInSearches 3) export a bunch of biblio records for zebra (e.g.: misc/migration_tools/rebuild_zebra.pl -I -b -x -k -length=1000), inspect the result xml records in /tmp/<whatever> file; observe that at the end of many records, here and there some extra "see from" (= 1st indicator: 'z') fields tend to appear, which shouldn't be there ;) To test: 4) apply patch 5) redo 3) 6) compare results from 3) and 5) with diff Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> I introduced a regression test for this. You should run the tests without/with the patch and verify that the patch actually fixes the problem. Good job Jacek! I'm sure writing the regression test would take less time than such a detailed commit message! Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>	9 years ago
Jonathan Druart	a6c9bd0eb5	Bug 9978: Replace license header with the correct license (GPLv3+) Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu> Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com> Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de> http://bugs.koha-community.org/show_bug.cgi?id=9987 Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>	9 years ago
Katrin Fischer	c5987b423b	Bug 9620: IncludeSeeFromInSearches and data in $9 subfields can break indexing In Germany 020$9 is used to store the ISBN with hyphens. In rare cases this can break indexing in combination with IncludeSeeFromInSearches. To reproduce: 1) Change your frameworks: * Make sure 084 $a is visible * Create a 084 $9 that you link to PERSO_NAME authorities. 2) Create a record or edit an existing one: * Use the plugin on 084 to link the record to an authority. * Make sure 400 is filled in your authority record. Preparations all done, now test indexing: 3) Make sure IncludeSeeFromInSearches is OFF * Reindexing should work without any problems. 4) Activate IncludeSeeFromInSearches * Reindexing should throw an error and stop: 201.Subfields are only for data fields (generally, just tags >= 010 at /home/koha/Koha/Filter/MARC/EmbedSee FromHeadings.pm line 94 5) Apply patch and reindex. * No error should occur. * Test 4xx fields from other tags were properly indexed. What causes the problem is the fact, that Koha tries to store the date from the 400 in the bibliographic record for indexing. For this it calculates a tag in the bibliographic record from the tag it found the authority link in and the 4xx tags used in the authority record. If you have an authority linked to a 0xx field and the matching authority has a 4xx where xx < 10, the calculated tag will be a control field. The attempt to add subfields to this control field, is causing the error and breaks the indexing process. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Comment: Work as described, and is a long description :) Test: 1) Enabled 084 (all subfields visible), added 084_9 (missing from default framework) linked to PERSO_NAME 2) Edited authority record and added some data to 400 3) Edited bib record, linked 084 to previous auth record 4) With IncludeSeeFromInSearches OFF, re-index without problem 5) With IncludeSeeFromInSearches ON, re-index problems!!! ==================== exporting biblio ==================== 1...Subfields are only for data fields (generally, just tags >= 010) at... Abort 6) Applied patch 7) With IncludeSeeFromInSearches ON, re-index without problem ... ==================== 5001.................................................................................................... Records exported: 5060 ==================== ... ==================== 2001.................................................................................................... Records exported: 2024 ==================== CLEANING ==================== Good! *) That's "works as described", don't know if other tests are needed. No errors from koha-qa Signed-off-by: Elliott Davis <elliott@bywatersolions.com> Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>	12 years ago
Jared Camins-Esakov	340ba63758	Bug 7417 follow-up: fix a perlcritic violation Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>	12 years ago
Jared Camins-Esakov	bc05b5d163	Bug 7417: Include see from references in bibliographic searches This patch adds the Koha::Indexer::RecordNormalizer and Koha::Indexer::MARC::RecordNormalizer::EmbedSeeFromHeadings packages to enable the inclusion of alternate forms of headings in bibliographic searches. When the new syspref IncludeSeeFromInSearches is turned on (default is off) rebuild_zebra.pl will insert see from headings from authority records into bibliographic records when indexing, so that a search on an obsolete term will turn up relevant records. To test: 1) Enable IncludeSeeFromInSearches 2) Add a heading that has an alternate form to a record (for example, "Cooking" has the alternate form "Cookery," if you have authority records from LC) 3) Index the zebraqueue (or reindex if you haven't indexed your system yet) 4) Confirm that if you search for "Cookery" you get the record you just modified Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com> Rebased on master 5 August 2012 Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com> Rebased on master 11 September 2012 Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Also checked: - Verified database update works correctly - Checked system preference and its description - Checked staff/opac detail pages with feature on/off - Checked staff/opac search facets - Downloaded and tested records in various formats - Tried different searches for 'see from' entries of authorities - Ran all unit tests No problems found.	12 years ago

25 Commits (dc79aa0f40f50f6c0d26fecf249b0b69fd3dc7ba)