On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
If a user to access a CSV profile that is not available at the OPAC we
should redirect to 404.
It will also handle undefined csv_profiles
For instance:
kohadev.mydnsname.org:8080/cgi-bin/koha/opac-downloadshelf.pl?shelfnumber=1&format=3
kohadev.mydnsname.org:8080/cgi-bin/koha/opac-downloadcart.pl?bib_list=1/2/&format=3
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch renames the introduced attribute for later usage on the API,
and changes the logic to use it the other way around.
It also adds a KEY for the flag, as it will be used in WHERE statements.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
To test:
1) Update database
2) Update schema
3) In staff side, go to Tools -> CSV profiles
4) Make a CSV profile with profile type = MARC and usage = Export
records
5) Leave 'Show option in OPAC' unchecked and save
6) Go to OPAC and add an item to your cart
7) Click Cart, click Download, confirm that your newly created CSV
profile does not show as an option
8) Go back and edit CSV profile on staff side
9) Check the 'Show option in OPAC' checkbox and save
10) Go back to download cart in OPAC
11) Confirm the CSV profile now shows in the dropdown
Sponsored-by: Catalyst IT
Signed-off-by: Maryse Simard <maryse.simard@inlibro.com>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
There is no need for all the conditions.
From Encode::encode POD:
"If the $string is undef, then undef is returned."
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
The cart was outputing ISO2709 MARC records with Latin-1
encoding. Records containing non-latin1 characters were
automatically re-encoded as UTF-8 by browsers, which led to
inconsistent character encodings for downloaded MARC files.
This patch explicitly encodes ISO2709 MARC characters from
the cart download as UTF-8 encoded bytes, which resolves the problem.
Test Plan:
0) Don't apply patch
1) Create bib record with only ASCII characters
2) Add a ü character to the title
3) Save bib record
4) Download bib record from cart (opac and staff client)
5) Using xxd or some other program, note that the ü is
represented by a FC byte (latin-1 encoded)
6) Apply the patch
7) Download bib record from cart (opac and staff client)
8) Using xxd or some other program, note that the ü is
represented by C3 BC bytes (utf-8 encoded)
9) Success
(Note that you could potentially use Notepad++ or some other
program to open the downloaded file and just note the encoding
that it finds. You could also try "chardetect" instead. Lots
of options for figuring out the encoding.)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
- Added missing GetHiddenItems parameter change case
Without this prove t had a failure.
- Always use mocks, not set_preference
- Tweaks so t/db_dependent/00-strict.t passes
There was a typo botcat vs borcat and borrowernumber was never
defined. Grabbing from userenv, like other code does.
- Tweak t/db_dependent/Items.t to fully test changes
This will test all the if structures fully in GetHiddenItemnumbers.
prove t/db_dependent/Items.t
- Tweak borrower category code
$borrower->{categorycode} on a Koha::Patron is not the
same as $borrower->categorycode. Fixed error.
- Search was returning URLS for wrong interface
There was one search context place wrong. Changed it to $is_opac
as the logic for setting $is_opac was modified correctly.
- Corrected issues with category code.
When a user isn't logged in, $borrower is undef and causes error
when determining category code. Added conditional check.
- Properly trigger all changes in C4/Search.pm
- Fix QA Test tool failures
C4/Search.pm had some tabs.
- Add some commenting to make sense of logic
- Refactor EmbedItemsInMarcBiblio parameters to hashref
- Trigger GetMarcBiblio's EmbedItemsInMarcBiblio call.
prove t/db_dependent/Items.t
- Add missing test to trigger Koha/BiblioUtils/Iterator change
- Add borrower category overrides
These files generally add borcat parameter to GetMarcBiblio.
Others might include correction of filtering of items
(opac-basket), or a comment as to why no changes were done
(opac-search).
In the case of opac-search, correcting the first FIXME will
likely correct the OpacHiddenItems issues on tags. As such,
that is beyond this bugs scope.
Some code had loop optimizations and fixes made, like a
'next unless $record' when the biblio shouldn't even be in
the list.
- Modify opac-ISBDdetail and opac-MARCdetail
Both files had similar logic. They were rearranged and
optimized, so that both files would have practically identical
initial blocks of code.
Optimizations were possible, because GetMarcBiblio
returns a filtered record, so that there is no double call
(once in the opac-### file and once in GetMarcBiblio) to
GetHiddenItemnumbers.
- Fix hiding in opac-tags
opac/opac-tags.pl was not properly hiding.
There is currently one known bug associated with tags left.
If you have two biblios tagged by different people with the
same tag, the opac-search will show the one you tagged that
is supposed to be hidden, because tag searches work differently
than regular searches. This is beyond the scope of this bug.
See the FIXME's in opac/opac-search.pl
- Trigger the C4::ILSDI::Services changes
prove t/db_dependent/ILSDI_Services.t
- Added missing 'my'
- Test C4/Labels/Label.pm changes
- Improve C4::Record::marcrecord2csv test cases
- Corrected opac-details searchResult call
- Fix breaking issues constraint in ITerator test
- Fix ILSDI_Services test when clubs with branch exist
- Rebased again!
- Rebased t/db_dependent/Items.t conflict.
The test plan is in comment #112 last I checked.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Change parameters to a hashref.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Looks good to me.
Two calls in migration_tools/22_to_30 still in old style.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds the frameworkcode option param, using each record's frameworkcode
as expected by the filter. Otherwise the ViewPolicy filter falls back to the
default framework.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Koha::RecordProcessor and the defined filters are supposed to bring us
joy and happiness. Let's keep the code compact, simple and clean.
This patch removes record cloning all over the place.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
There are still some leaks, but it is not as a result
of the filter, but rather a result of poorly written
template files.
Bug fixing template files is beyond the scope of this
set of patches.
TEST PLAN
---------
1) Backup your DB
2) run the following SQL on your DB.
> UPDATE marc_subfield_structure set hidden=-8;
-- this should set EVERYTHING to hidden across the board.
3) In staff client, set OPACXSLTDetailsDisplay to blank
4) In OPAC, view any detail.
-- Normal view may mostly leak values still.
-- MARC view may leak values.
-- ISBD view may leak values.
5) In staff client, set OPACXSLTDetailsDisplay to default
6) In OPAC, view any detail.
-- same issues as step 4
-- 'View Plain' may leak too.
7) 'Save record' -> 'Dublin Core'
8) Apply this patch
9) run koha qa test tools
-- should be fine
10) prove -v t/db_dependent/Filter_MARC_ViewPolicy.t
-- should pass
-- this proves Koha/Filter/MARC/ViewPolicy.pm tweaks too
11) In OPAC, view any detail.
-- Normal view:
-- Material type comes from the LEADER field.
-- Lists this is on will still display
-- 'Tags from this library' will still display
-- Item information in table will still display
(THIS IS BEYOND SCOPE)
-- MARC view:
-- Record number is leaked
(THIS IS BEYOND SCOPE)
-- 'View plain' leaks LEADER field.
-- ISBD view may leak field headings, but not values.
(THIS IS BEYOND SCOPE)
12) In staff client, set OPACXSLTDetailsDisplay to blank
13) In OPAC, view any detail.
-- same kind of output as step 10
14) 'Save record' -> BIBTEXT
-- Should be next to nothing leaked.
15) 'Save record' -> Dublin Core
-- Should be the same or less leaked between the two versions.
-- (XML FILTERING IS BEYOND SCOPE)
16) In the staff client, go view the same record.
-- it should be mostly hidden in ISBD View.
17) run the following SQL on your DB.
> UPDATE marc_subfield_structure set hidden=1;
-- this should set EVERYTHING to hidden in OPAC, but not
the STAFF across the board.
18) Refresh the staff ISBD page
-- values should reappear.
19) View the ISBD details in the OPAC
-- values should still be hidden.
20) Check out the OPAC Cart and List
-- while the intermediate pages may still leak
the download links should leak very minimally.
-- (CARTS AND LISTS ARE BEYOND SCOPE, THOUGH
THE INTRANET ISBD AND SOME CART/LIST STUFF
WERE FIXED BECAUSE OF THE GetISBDView REFACTOR)
Expectations:
Before Patch - all the OPAC Detail pages will display things
After Patch - all the OPAC Detail pages will display much less,
and hopefully nothing (though there are known limits).
the ISBD detail page in the Staff client will be
filtered as well based on STAFF settings.
The saving/exporting should generate nearly empty
files.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch erase all traces of C4::Csv since it's not used anymore.
All occurrences have been replaced by previous patches to use
Koha::CsvProfiles.
Note that GetMarcFieldsForCsv was not used prior this patch set.
Test plan:
git grep 'C4::Csv'
should not return any result.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
No more traces of the file.
This produces a koha-qa fail, due to the missing file.
No other errors
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This subroutine returned the csv profiles for a given type.
This could be done easily with the new Koha::CsvProfiles->search method.
Test plan:
To do at the OPAC and staff interface!
1/ Export a list using a CSV profile
2/ Export your CART using a CSV profile
Note that only MARC profiles should be available.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Tested on staff/opac & cart/list
Small problem on filename extension fixed in followup.
No errors
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The borrow permission was used but uselessly.
For instance, at the opac, the flagsrequired parameter was set to
'borrow' but the 'authnotrequired' was set also (which means no auth
required).
At the end, this permission was used at only 1 place: for the basket,
intranet side.
This can be replaced with the catalogue permission (which is used to
search).
Test plan:
1/ Confirm that you are able to show/download/sent the cart (intranet side)
with the catalogue permission.
2/ At the OPAC, you should be able to access the same pages as before
with any other permissions.
Concretely it is quite difficult to test this patch, you should have a
look at the code.
Signed-off-by: Nick Clemens <nick@quecheelibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This patch adds the option to download records from the cart, and
lists in the ISBD format from the OPAC.
To test (cart):
- Apply the patch
- Add several records to your OPAC cart.
- Go to your cart, and choose 'Download'
=> SUCCESS: There's an ISBD option
- Dowload and open the exported records
=> SUCCESS: The file contains the ISBD format for the records on the cart
To test (lists):
- Add several records to a list (i did it from the cart ;-))
- Open the list
- Choose 'Download list'
=> SUCCESS: There's an ISBD option
- Download and open the exported records
=> SUCCESS: The file contains the ISBD format for the records on the list.
- Sign off :-D
Sponsored-by: Orex Digital
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Since we switched to Template Toolkit we don't need to stick with the
sufix we used for HTML::Template::Pro.
This patch changes the occurences of '.tmpl' in favour of '.tt'.
To test:
- Apply the patch
- Install koha, and verify that every page can be accesed
Regards
To+
P.S. a followup will remove the glue code.
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Test plan:
Verify that existing CSV lists list MARC CSV profiles and not SQL CSV
profiles.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
- The following export pages used to embed items when exporting,
this was no longer the case, so they were fixed :
Intranet :
- basket/downloadcart.pl,
- virtualshelves/downloadshelf.pl
- catalogue/export.pl
Opac :
- opac/opac-downloadcart.pl
- opac/opac-downloadshelf.pl
- opac/opac-export.pl
- Notes :
- GetMarcBiblio used to embed items data, this was no longer the case,
so an optional parameter was added to choose if items should be embedded or not.
This way, previous work on this bug is not broken, and this is a pretty usefull
feature, imho.
- An optional parameter has been added to SetUTF8Flag, to be able to use NFD during
normalization. This was required to make Unicode/UTF-8 export work again.
Signed-off-by: Claire Hernandez <claire.hernandez@biblibre.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Use of switch generates errors in perl 12 and above
Replaced uses of the Switch module by if- else constructs
Removed some mixed space+tab indentation around these where
it generated visual confusion
As of perl 10 you can use given/when constructs but this is not
supported in earlier perls.
On a basket or a basket download, or csv export, if Koha cannot get a biblio, then it failed with error 500.
This patch fixes that behaviour skipping the faulty record in order to present the user with the biblios which are not causing trouble.
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Adds the ability to customize CSV exports through the use of a YAML file.
The following customizations are available :
- Preprocessing
- Postprocessing
- Field-by-field processing
The YAML field should be stored in the tools/csv-profiles/ directory and
named after the id of the CSV profile you want to customize.
An example file is provided in that directory.
(cherry picked from commit 76655b5b94)
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
