In current implementation (mostly commented out in this patch)
uses heuristic to guess which strings need decoding from utf-8
to binary representation and doesn't support utf-8 characters
in templates and has problems with utf-8 data from database.
With this changes, Koha perl code always uses utf-8 encoding
correctly. All incomming data from database is allready
correctly marked as utf-8, and decoding of utf8 is required
only from Zebra and XSLT transfers which don't set utf-8 flag
correctly.
For output, standard perl :encoding(utf8) handler is used
so it also removes various "wide character" warnings as side-effect.
Test scenario:
1. make sure that you have utf-8 characters in your biblio
records, patrons, categories etc.
2. try to search records on intranet and opac which contain
utf-8 characters
3. install language which has utf-8 characters, e.g. uk-UA
dpavlin@koha-dev:/srv/koha/misc/translator(bug_6554) $
PERL5LIB=/srv/koha/ perl translate install uk-UA
4. switch language to uk-UA and verify that templates
display correctly
5. test search and Z39.50 search and verify that caracters
are correct
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
I followed the test plan, adding utf-8 characters to library names,
patron categories, titles, and authorized values. I tried the uk-UA
translation and everything looked good.
When performing Z39.50 searches for titles containing utf-8 characters I
got results which were still occasionally contaminated with dummy
characters [?] but I assume this is Z39.50's fault not the patch's.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Already signed, add mine.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
- Filter by groups using tabs
- Filter by subgroups using a dropdown list under each tab
- Allow to create new group and subgroups when building a new report or
editing an existing report (no need to go through admin AV interface)
+ Possibility to duplicate a report (new link in the table)
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This patch adds the ability to select the number of row to show per page while
retaining the default rows per page as 20.
Test Plan:
1) Apply patch
2) Run a guided report that will have many resultant rows
3) Try the various rows per page options
4) Verify the rows per page selected is retained when paging through results
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
- the dateformat value is send to all templates (from
C4::Auth::get_template_and_user)
- remove all assignment of dateformat in all .pl files
- the DHTMLcalendar_dateformat variable is unused
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Fixed conflicts:
- opac/sco/sco-main.pl
- reports/acquisitions_stats.pl
- tools/cleanborrowers.pl
All tests pass, perlcritic problems appeared in some files
before and after these patches were applied.
Checked sorting in following pages:
- acqui/addorderiso2709.tt - list of staged imports in acq
- acqui/histsearch.tt - sorting of dates in acq search result list
- acqui/invoices.tt - billing date in list of invoices in acq
- acqui/lateorders.tt - list of late orders in acq
- acqui/ordered.tt - ordered titles and estimated costs for a fund
- acqui/parcels.tt - receive shipment page
- acqui/spent.tt - received titles and actual costs for a fund
...
- serials-search.tt - subscription search result list
...
- opac/sco/sco-main.tt - due dates in list of checked out items
- reports/acquisitions-stats.tt - date searches, display of dates
- tools/cleanborrowers.tt
- tools.holidays.tt - different views of dates library is closed,
adding dates
Checked dates display according to system preference everywhere and
searching, entering dates etc. still worked as expected.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
The fix for Bug 7993 removed some lines from guided_reports.pl
which passed a new report's name and id to the template after
saving. For some reason this was removed, making it impossible
to run or edit a new report immediately after saving.
This patch adds back that part of the script which passes those
variables to the template. Before this patch, after adding a new
SQL report you would not see a "run this report" link. The
"run this report" and "edit" buttons in the toolbar would not
work. After applying the patch these should work.
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Tested creating reports and was able to run them after saving.
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This should make saved reports more manageable.
Group/Subgroup hierarchy is stored in authorised_values,
categories REPORT_GROUP and REPORT_SUBGROUP, connected by
REPORT_SUBGROUP.lib_opac -> REPORT_GROUP.authorised_value
Database changes:
* authorised_values: expanded category to 16 chars
* created default set of REPORT_GROUP authorised values to match
hardcoded report areas
* reports_dictionary: replaced area int with report_area text, converted
values
* saved_sql: added report_area, report_group and report_subgroup;
report_area is not currently used, saved for the record
C4/Reports/Guided.pm:
* Replaced Area numeric values with the mnemonic codes
* get_report_areas(): returns hardcoded areas list
* created get_report_areas(): returns full hierarchy (groups with belonging
subgroups)
* save_report(): changed iterface, accepts fields hashref as input
* update_sql(): changed iterface, accepts id and fields hashref as input
* get_saved_reports():]
- join to authorised_values to pick group and subgroup name
- accept group and subgroup filter params
* get_saved_report():
- changed iterface, return record hashref
- join to authorised_values to pick group and subgroup name
* build_authorised_value_list(): new sub, moved code from
reports/guided_reports.pl
* Updated interfaces in:
cronjobs/runreport.pl, svc/report, opac/svc/report: get_saved_report()
reports/dictionary.pl: get_report_areas()
reports/guided_reports.pl
reports/guided_reports_start.tt:
* Reports list:
- added group/subgroup filter
- display area/group/subgroup for the reports
* Create report wizard:
- carry area to the end
- select group and subgroup when saving the report; group defaults to area,
useful when report groups match areas
* Update report and Create from SQL: added group/subgroup
* Amended reports/guided_reports.pl accordingly
Conflicts:
C4/Reports/Guided.pm
admin/authorised_values.pl
installer/data/mysql/kohastructure.sql
installer/data/mysql/updatedatabase.pl
koha-tmpl/intranet-tmpl/prog/en/modules/reports/dictionary.tmpl
koha-tmpl/intranet-tmpl/prog/en/modules/reports/guided_reports_start.tmpl
misc/cronjobs/runreport.pl
reports/dictionary.pl
reports/guided_reports.pl
Signed-off-by: Delaye Stephane <stephane.delaye@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This reverts commit 215abc8024.
The 3 patches for bug 8089 have been reverted, because they break
jenkins & Koha.
A follow-up has been provided, but it does not solve the problem on my
test server, it just changes the error message.
After a discussion with jared, Dobrica should work on another patch, so
the best option is to revert.
1. Replace all instances of memoize_memcached with appropriate calls
into Koha::Cache:
* reports/guided_reports.pl
* C4::Biblio::GetMarcStructure
* C4::Languages::getFrameworkLanguages
* C4::Languages::getAllLanguages
* C4::SQLHelper::GetPrimaryKeys
* C4::SQLHelper::_get_columns
2. Replace all references to memcached with the appropriate calls into
Koha::Cache in C4::Context.
Test plan :
* have DEBUG env set to 1
* reach addbiblio page to test the patch in Biblio.pm, or setup more than 1
language
* you should see in the logs that you're reading and writing from cache
* run the test suite twice both with and without the following environment
variables set:
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
export CACHING_SYSTEM=memcached
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
I'm unsure about some of the caching times 10000 is a long long time,
but other than that, works fine.
Changes to markup and script to enable unique ids on form
fields. These changes allow for working label+input pairs
on one or more text and select inputs and multiple working
datepickers.
To test, create a report with more than one date parameter,
at least one text parameter, and one or more authorised
value parameters. The datepickers should each work properly,
and labels should by clickable.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Hands back JSON, and supports caching
Squashed commit of the following:
commit 5c93506079738b9a13139417d3a0734289a4b007
Author: Chris Hall <chrish@catalyst.net.nz>
Date: Mon Nov 21 15:41:14 2011 +1300
Fixed bugs when creating reports, added more user friendly input and
feedback
commit c62430bb358ee4af6ee5331b4d3a2ed67f723032
Author: Chris Hall <chrish@catalyst.net.nz>
Date: Mon Nov 21 12:00:27 2011 +1300
Added caching to intranet report webservices, added cache expiry and public
options to reports
commit 2d89f0777d95b26bf08635782070b6367d0698f3
Author: Chris Cormack <chris@bigballofwax.co.nz>
Date: Wed Nov 9 20:29:26 2011 +1300
Working on caching for services
commit e0511f180cebd81747858ad776433fe3a1cf6854
Author: Chris Cormack <chris@bigballofwax.co.nz>
Date: Wed Nov 9 08:28:26 2011 +1300
Starting work on webservices
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Patch includes database update adding 2 new fields to the saved_sql table.
1) Checked that adding/deleting/editing of sql reports still works and new
fields are correctly saved to the database.
2) The saved reports page now lists the new fields correctly.
3) Checking URLs with JSON output
- if report is not public http://localhost/cgi-bin/koha/svc/report?id=2 fails,
but http://localhost:8080/cgi-bin/koha/svc/report?id=2 works
- if report is public, both links work
4) Checking results are cached
Created a report listing my borrowers, setting caching to 60 seconds and
added new borrowers before doing following tests.
- URL doesn't show new borrower immediately - correct
- Turning off the usecache system preference updates data - correct
- Restarting memcached updates data '/etc/init.d/memcached restart' - correct
- Waiting until cache time runs out updates data - correct
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Retested the feature and renewing my sign-off.
There are some things that should be noted:
- currently there is a hardcoded limit of 10 results for the reports webservice
- reports with runtime parameters are currently not supported
Creating new include, reports-toolbar.inc for presenting "action"
options to the user, in contrast with "view" options in the left-
hand navigation menu.
In the toolbar: New (guided report, SQL report), Edit, Run.
The presence of the toolbar allows the user to access functions
more flexibly: Getting directly to 'edit' or 'run' from the 'view'
page for instance.
Modifications to guided_reports.pl pass report id and name to
the template for clarity and for the purpose of enabling the
edit/run buttons.
To test: Apply the patch and go through the process of creating
a new saved SQL report. Note that the toolbar is present and
the buttons are functional at appropriate times.
New and Edit options should only be displayed if the user has
permission to create reports. Test with a user who does not
have create permission to confirm.
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Adds a date picker for SQL reports with the 'date' authorised_value. Updates help
documentation for this new feature, as well.
To test:
1. Create a new report with two date parameters, using the syntax:
<<Label for this selection of date|date>>
2. Run the report
3. Select dates
4. The SQL generated should format the dates properly in ISO, and keep track of which
date is which
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
use encoding(UTF-8) rather than utf-8 for stricter
encoding
Marking output as ':utf8' only flags the data as utf8
using :encoding(UTF-8) also checks it as valid utf-8
see binmode in perlfunc for more details
In accordance with the robustness principle input
filehandles have not been changed as code may make
the undocumented assumption that invalid utf-8 is present
in the imput
Fixes errors reported by t/00-testcritic.t
Where feasable some filehandles have been made lexical rather than
reusing global filehandle vars
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Fixes bug where paging through output of an SQL report that includes
one or more parameters fails by asking you for the parameter values
and again and sending you back to the first page of results.
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Patch fixes a TT error that caused Build New to be empty.
To test:
Click More -> Reports, you should see the welcome page and be able to step through creating the report.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
1. Moved the form to the left under the menu
2. Added a calendar next to date (and a date format tip)
3. Styled the form like other filter forms for consistency's sake
4. Formatted on-screen message to match standard style
Signed-off-by: Nicole C. Engard <nengard@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: fdurand <frederic.durand@univ-lyon2.fr>
Signed-off-by: Ian Walls <ian.walls@bywatersolutions.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
More podchecker cleanups to eliminate warnings / errors
Signed-off-by: Andrew Elwell <Andrew.Elwell@gmail.com>
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
* Enforce the requirement that the user must have the
create_reports permission in order to delete a saved report;
closes hole where unprivileged user could delete reports
by constructing a URL maliciously
* Added another tweak of the template - don't offer option
to create a new report if the user doesn't have permission.
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
missing s
create_reports
and execute_reports
rather than
create_report
and execute_report
Signed-off-by: Galen Charlton <gmcharlt@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
added runtime parameters for guided reports. Online help:
Can I have run-time defined parameters?
Yes, you can: there is a specific syntax that Koha will understand as 'ask for values when running the report'. The syntax is <<Question to ask|authorised_value>>.
The << and >> are just delimiters. You must put << at the beginning and >> at the end of your parameter
The Question to ask will be displayed on the left of the string to enter.
The authorised_value can be omitted if not applicable. If it contains an authorised value category, or <strong>branches</strong> or <strong>itemtype</strong> or <strong>categorycode</strong>, a list with the Koha authorised values will be displayed instead of a free field
Note that you can have more than one parameter in a given SQL
Note that entering nothing at run time won't probably work as you expect. It will be considered as "value empty" not as "ignore this parameter". For example entering nothing for : "title=<<Enter title>>" will display results with title='' (no title). If you want to have to have something not mandatory, use "title like <<Enter title>>" and enter a % at run time instead of nothing
Sample :
SELECT surname,firstname FROM borrowers WHERE branchcode=<<Enter patrons library|branches>> AND surname like <<Enter filter for patron surname (% if none)>>
This patch add 2 columns in "Use Saved" page whith creation date, and the creator id.
It fix the creation of a report, to save the borrowernumber in reports table(to show it after).
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Previous patch broke this, always providing the
guided report download in CSV format regardless of
whether CSV or tab-delimited was requested.
Also, default name of downloaded file is reportresults.csv
for CSV output and reportresults.tab for tab-delimited.
Also removed a couple subroutine prototypes that
were leading to unnecessary compile-time warnings
under use warnings.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
execute_query now refactored, returns reliable results, does
zero presentation-layer crap. Arguments reduced, client scripts
adapted to new API and performance improved. Text::CSV now used
to generate CSV output, ensuring portability, encoding and accuracy.
Replaced tools/runreport.pl with misc/cronjobs/runreport.pl:
~ security fixed
~ documentation improved
~ TODO: finish sendmail option.
Bug 3077 also fixed.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
Guided reports and dictionary access was dependant on the editcatalogue permission instead of the report permission.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
This repurposes the Reports module for general reports, and moves its functionality to a different module, Reports::Guided. It also updates all of the reports that can export to correctly treat the 'tabulation' value of the 'delimiter' syspref.
Signed-off-by: Galen Charlton <galen.charlton@liblime.com>
