This alters the svc scripts to set the report id after fetchign the report object
to ensure it is passed to exectue query
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
The last run of a report is updated only if method execute_query() is
called with report_id.
This whas missing for :
- when report is run publicly
- when report is sent by email
- when report is exported
Patch changes the method signature to use a hash of params, in order to
easily avoid some params.
Test plan :
1) Create a report.
2) Run report.
3) Check the report listing. Confirm that the last run info on the report is updated.
4) Make report public.
5) Run report via public url.
6) Check the report listing. Confirm that the last run info on the report IS NOT updated.
7) Schedule the report to run at a given time and e-mailed to an address.
8) After the report runs at the scheduled time, check the report listing. Confirm that the last run info on the report IS NOT updated.
9) Run report.
10) Export results.
11) Check the report listing. Confirm that the last run info on the report IS NOT updated AT THE TIME OF THE EXPORT.
Questionable (I don't know if this is addressed):
12) Run report on backend through a cron job and send results via e-mail.
13) Check the report listing. Confirm that the last run info on the report IS NOT updated.
14) Apply patch.
15) Rerun steps 2-13. Confirm that steps 3, 6, 8, 11, and 13 DO UPDATE the last run info.
Signed-off-by: Séverine Queune <severine.queune@bulac.fr>
Signed-off-by: Séverine Queune <severine.queune@bulac.fr>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
TO test:
1 - Write a report
2 - Click the svc link in the saved reports list
3 - ISE
4 - Make report public
5 - Click the svc link in saved reports list
6 - ISE
7 - Apply patch
8 - Comfirm opac/public link works
9 - Make report not public
10 - Confirm the internal svc link works
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch makes the opac/svc/report script use output_with_http_headers
so it sets the Access-Control-Allow-Origin header.
To test:
1. Create a new public report and remember the report id
2. Use your favourite too to fetch the report in JSON by issuing:
GET http://localhost:8080/cgi-bin/koha/svc/report?id=1
=> FAIL: There is no Access-Control-Allow-Origin header
3. Apply this patch
4. Restart Plack
$ sudo koha-plack --restart kohadev
5. Set the AccessControlAllowOrigin to anything (for example,
https://koha-community.org)
6. Repeat 2
=> SUCCESS: On the response headers you find
Access-Control-Allow-Origin: https://koha-community.org
7. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds the ability to rename columns in MySQL reports using a syntax:
[[column|alias]]
To test:
1 - Write a report like:
SELECT cardnumber FROM borrowers WHERE cardnumber LIKE '%9'
2 - Run report
3 - Note you can batch modify the patrons
4 - Update report:
SELECT cardnumber AS library_barcode FROM borrowers WHERE cardnumber LIKE '%9'
5 - Run report
6 - Note batch options are no longer present
7 - Apply patch
8 - Run report, no change
9 - Update report like:
SELECT [[cardnumber|library_barcode]] FROM borrowers WHERE cardnumber LIKE '%9'
10 - Run report
11 - Batch options work!
12 - Update report like:
SELECT cardnumber FROM borrowers WHERE cardnumber LIKE '%9'
13 - Verify batch options still work
14 - Set report 'public'
15 - Verify report can be run via JSON URLs:
http://kohadev.mydnsname.org:8080/cgi-bin/koha/svc/report?id=1http://kohadev.mydnsname.org:8081/cgi-bin/koha/svc/report?id=1
Signed-off-by: George Williams <george@nekls.org>
Signed-off-by: George Williams <george@nekls.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Resolve things like:
CGI::param called in list context from package CGI::Compile::ROOT::usr_share_koha_prodclone_opac_svc_report line 42, this can lead to vulnerabilities. See the warning in "Fetching the value or values of a single named parameter" at /usr/share/perl5/CGI.pm line 436.
The cache key in both script looks like:
opac:report:id:602018
but should for consistency be:
opac:report:id:60:2018
Note: The 2018 here is part of the sql_params and should not be
concatenated to the report id.
Test plan:
Do not yet apply this patch.
Make a report public, set cache to 300 secs.
Check its output with opac/svc/report.
Check for the warn in your log.
Apply the patch, restart Plack and flush cache.
Check opac/svc/report.
Modify your report; e.g. add a simple string to the SELECT.
Check opac/svc/report. You should still see cached output.
Flush the cache.
Check opac/svc/report. You should now see the added text.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested also by clearing individual keys with $cache->clear_from_cache.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Ultimately we should probably remove name access as it is not a unique
id, but this should preserve existing behaviour
To test:
Create a report
Use the service link to confirm the report runs
Replace id=# parameter with name=XXXXXX
Confirm URL works
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Note: We should not remove the param "name", it's a feature, see bug 8256.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
To test:
1 - prove t/db_dependent/Reports/Guided.t
2 - grep "get_saved_report" - ensure there are no occurences of the
singular form
3 - create, save, edit, and convert a report
4 - access a public report and report json from opac and staff client
5 - Ensure all function as expected
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Cab Vinton <director@plaistowlibrary.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
s/Koha::Cache->get_instance/Koha::Caches->get_instance
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
https://bugs.koha-community.org/show_bug.cgi?id=11921
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This makes Koha::Cache behave better by default. It will use memcached
if available to do shared caching, if that's not available it will fall
back to in-memory caching. It also allows for a singleton accessor to
allow a single cache to be shared within a process.
* Added tests to confirm UTF8-cleanness.
* Added minor fixups to stop warnings.
Test plan:
* The t/Cache.t file runs successfully with and without the
MEMCACHED_SERVERS envvar set (and memcached running in the
environment.)
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
- catalog a record that includes umlauts äöü
- write a report, that has the record in the result set
- access the JSON output of the report
<OPAC BaseURL/cgi-bin/koha/svc/report?id=1
- verify the umlaut displays not correctly
- switch the encoding in the browser to 'western'/latin1
- verify the umlaut now displays correctly
- apply patch
- verify umlaut now displays correctly from the beginning
Based on paste from Galen Charlton.
Followed test plan. Result as expected.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This patch teaches C4::Reports::Guided::execute_query()
how to accept a list of query parameter values. It then
follows-up on the main patch by simplifying how it converts
report parameters to a complete SQL query, and removes the
use of DBI->quote() and complicated regexes.
To test:
[1] Verify that using the OPAC svc/report service with
sql_params continues to work.
[2] Verify that there are no regressions with running
reports from the staff interface, both via the web
service and the reports interface.
[3] Verify that prove -v /db_dependent/Reports_Guided.t passes.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
No regressions found.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Test plan:
1/ Make a public report that contains SQL parameters
2/ Test it from the staff side
3/ Hit the url like
cgi-bin/koha/svc/report?id=1&sql_params=chris&sql_params=chris2
4/ Test you get JSON
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Tested with a public report:
select * from items
- cgi-bin/koha/svc/report?id=6
- URL shows JSON and all items
select * from items where itype = <<itype>>
- cgi-bin/koha/svc/report?id=6&sql_params=BK
- URL shows JSON and only BK items
select * from items where itype = <<itype>> and homebranch = <<homebranch>>
- cgi-bin/koha/svc/report?id=6&sql_params=BK&sql_params=MPL
- URL shows JSON and only BK items in MPL
select * from items where itype = <<itype|itemtypes>> and homebranch = <<homebranch|branches>>
- same as above
- URL shows JSON and only BK items in MPL
select * from items where dateaccessioned = 2013-12-15
- cgi-bin/koha/svc/report?id=6&sql_params=BK&sql_params=MPL&sql_params=2013-12-15
- URL shows JSON and correct items
Note: This will currently only work for public reports!
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
To test:
1 - Go through the first comments instructions to reproduce
and verify the bug is present (OPAC and STAFF)
2 - Apply the patch
3 - Repeat step 1 and notice
* characters are not broken anymore
* the header is correct
4 - Sign off
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Replaced License text with latest one from
http://wiki.koha-community.org/wiki/Coding_Guidelines#Licence
and tweaked copyright lines accordingly.
Also changed use strict and use warnings into use Modern::Perl
as per PERL2 from the coding guidelines.
TEST PLAN
---------
1) less ~/kohaclone/opac/svc/report
- The license does not reflect the current license
- This will have use strict and use warnings
2) Apply patch
- The license should be corrected
3) It should still run
https://.../cgi-bin/koha/svc/report?id=##https://.../cgi-bin/koha/svc/report?id=##&annotated=blah
Where ## reflects a public report available (for OPAC).
Signed-off-by: Holger Meißner <h.meissner.82@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
The staff and public catalog reports web services (/svc/report) return
JSON output, in particular an array contain an array for each row
of the report output.
This patch adds a URL parameter, annotated, which when supplied and
set to a value that evaluates to Perl true, will cause the output
to be emited as a JSON array of hashes, where each hash represents
a row of report output and the hash keys are the column names
or labels set in the report definition.
This patch also moves code around to make diffs between svc/report and
opac/svc/report smaller.
The suggestion to return an array of hashes was made by Chris Cormack.
TEST PLAN
---------
1) Log into staff client
2) Reports
3) Used save reports
4) Click the Action button on any report
WITHOUT PARAMETERS.
-- with parameters blows up in master and this.
5) Click Edit
6) Make the report public
7) Update the SQL
8) Note the ID number of the report
9) Note the ID number of a non-public report
10) Make up a crazy ID number for a non-existant report
11) In a new tab (with the appropriate edits)
https://OPAC/cgi-bin/koha/svc/report?id=# from step 8
-- JSON data has arrays of field values.
https://OPAC/cgi-bin/koha/svc/report?id=# from step 8&annotated=1
-- JSON data has arrays of field values.
https://OPAC/cgi-bin/koha/svc/report?id=# from step 9
-- Software error: Sorry this report is not public
https://OPAC/cgi-bin/koha/svc/report?id=# from step 9&annotated=1
-- Software error: Sorry this report is not public
https://OPAC/cgi-bin/koha/svc/report?id=# from step 10
-- Software error: Sorry this report is not public
https://OPAC/cgi-bin/koha/svc/report?id=# from step 10&annotated=1
-- Software error: Sorry this report is not public
https://STAFF/cgi-bin/koha/svc/report?id=# from step 8
-- JSON data has arrays of field values.
https://STAFF/cgi-bin/koha/svc/report?id=# from step 8&annotated=1
-- JSON data has arrays of field values.
https://STAFF/cgi-bin/koha/svc/report?id=# from step 9
-- JSON data has arrays of field values.
https://STAFF/cgi-bin/koha/svc/report?id=# from step 9&annotated=1
-- JSON data has arrays of field values.
https://STAFF/cgi-bin/koha/svc/report?id=# from step 10
-- Software error: hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this) at /usr/share/perl5/JSON.pm line 154.
https://STAFF/cgi-bin/koha/svc/report?id=# from step 10&annotated=1
-- Software error: hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this) at /usr/share/perl5/JSON.pm line 154.
12) Apply the patch
13) ~/qa-test-tools/koha-qa.pl -v 2 -c 2
-- There should be no problems. All OK.
14) In a new tab (with the appropriate edits)
https://OPAC/cgi-bin/koha/svc/report?id=# from step 8
-- JSON data has arrays of field values.
https://OPAC/cgi-bin/koha/svc/report?id=# from step 8&annotated=1
-- JSON data has arrays of hashes with field names as keys
https://OPAC/cgi-bin/koha/svc/report?id=# from step 9
-- Software error: Sorry this report is not public
https://OPAC/cgi-bin/koha/svc/report?id=# from step 9&annotated=1
-- Software error: Sorry this report is not public
https://OPAC/cgi-bin/koha/svc/report?id=# from step 10
-- Software error: There is no such report.
https://OPAC/cgi-bin/koha/svc/report?id=# from step 10&annotated=1
-- Software error: There is no such report.
https://STAFF/cgi-bin/koha/svc/report?id=# from step 8
-- JSON data has arrays of field values.
https://STAFF/cgi-bin/koha/svc/report?id=# from step 8&annotated=1
-- JSON data has arrays of hashes with field names as keys
https://STAFF/cgi-bin/koha/svc/report?id=# from step 9
-- JSON data has arrays of field values.
https://STAFF/cgi-bin/koha/svc/report?id=# from step 9&annotated=1
-- JSON data has arrays of hashes with field names as keys
https://STAFF/cgi-bin/koha/svc/report?id=# from step 10
-- Software error: There is no such report.
https://STAFF/cgi-bin/koha/svc/report?id=# from step 10&annotated=1
-- Software error: There is no such report.
Signed-off-by: Holger Meißner <h.meissner.82@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
This should make saved reports more manageable.
Group/Subgroup hierarchy is stored in authorised_values,
categories REPORT_GROUP and REPORT_SUBGROUP, connected by
REPORT_SUBGROUP.lib_opac -> REPORT_GROUP.authorised_value
Database changes:
* authorised_values: expanded category to 16 chars
* created default set of REPORT_GROUP authorised values to match
hardcoded report areas
* reports_dictionary: replaced area int with report_area text, converted
values
* saved_sql: added report_area, report_group and report_subgroup;
report_area is not currently used, saved for the record
C4/Reports/Guided.pm:
* Replaced Area numeric values with the mnemonic codes
* get_report_areas(): returns hardcoded areas list
* created get_report_areas(): returns full hierarchy (groups with belonging
subgroups)
* save_report(): changed iterface, accepts fields hashref as input
* update_sql(): changed iterface, accepts id and fields hashref as input
* get_saved_reports():]
- join to authorised_values to pick group and subgroup name
- accept group and subgroup filter params
* get_saved_report():
- changed iterface, return record hashref
- join to authorised_values to pick group and subgroup name
* build_authorised_value_list(): new sub, moved code from
reports/guided_reports.pl
* Updated interfaces in:
cronjobs/runreport.pl, svc/report, opac/svc/report: get_saved_report()
reports/dictionary.pl: get_report_areas()
reports/guided_reports.pl
reports/guided_reports_start.tt:
* Reports list:
- added group/subgroup filter
- display area/group/subgroup for the reports
* Create report wizard:
- carry area to the end
- select group and subgroup when saving the report; group defaults to area,
useful when report groups match areas
* Update report and Create from SQL: added group/subgroup
* Amended reports/guided_reports.pl accordingly
Conflicts:
C4/Reports/Guided.pm
admin/authorised_values.pl
installer/data/mysql/kohastructure.sql
installer/data/mysql/updatedatabase.pl
koha-tmpl/intranet-tmpl/prog/en/modules/reports/dictionary.tmpl
koha-tmpl/intranet-tmpl/prog/en/modules/reports/guided_reports_start.tmpl
misc/cronjobs/runreport.pl
reports/dictionary.pl
reports/guided_reports.pl
Signed-off-by: Delaye Stephane <stephane.delaye@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Adds the ability to pass a hash to C4::Reports::Guided::get_saved_report
which specifies a name or id to select the report.
Test plan:
1. Create a report (or choose an existing one), and mark it public
2. Run the report using the web service: [IntranetBaseURL]/cgi-bin/koha/svc/report?id=whatever
3. Confirm you get the expected results
4. Run the report by name using the web service: [IntranetBaseURL]/cgi-bin/koha/svc/report?name=[Report name] (keep spaces in the name)
5. Confirm you get the same results
6. Run the report using the public web service: [OPACBaseURL]/cgi-bin/koha/svc/report?id=whatever
7. Confirm you get the same results
8. Run the report by name using the public web service: [OPACBaseURL]/cgi-bin/koha/svc/report?name=[Report name] (keep spaces in the name)
9. Confirm you get the same results
10. Sign off
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
- Adding, editing and deleting reports works
- id parameter works
- new name parameter works
- public and non-public works
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
This patch introduces some new features for caching system:
* the type of caching is retrieved from ENV variable in httpd configuration if not forced when the ->new() is called.
* if $ENV{DEBUG} is ON, you'll have feedback when something is set or read from cache
* the Koha::Cache->is_cache_active is now available and will return 1 if there is a caching system available. It's a replacement for C4::Context->ismemcached
* fixes the mistake for debug/compress_threshold and expire_time parameters
The 2 report web service modules have been updated to use this new API
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
The original patches for bug 7249 had a hardcoded limit of ten rows in the
results returned by the web service. This patch adds a SvcMaxReportRows syspref
which allows the librarian to choose how many rows should be returned by the
web service.
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Hands back JSON, and supports caching
Squashed commit of the following:
commit 5c93506079738b9a13139417d3a0734289a4b007
Author: Chris Hall <chrish@catalyst.net.nz>
Date: Mon Nov 21 15:41:14 2011 +1300
Fixed bugs when creating reports, added more user friendly input and
feedback
commit c62430bb358ee4af6ee5331b4d3a2ed67f723032
Author: Chris Hall <chrish@catalyst.net.nz>
Date: Mon Nov 21 12:00:27 2011 +1300
Added caching to intranet report webservices, added cache expiry and public
options to reports
commit 2d89f0777d95b26bf08635782070b6367d0698f3
Author: Chris Cormack <chris@bigballofwax.co.nz>
Date: Wed Nov 9 20:29:26 2011 +1300
Working on caching for services
commit e0511f180cebd81747858ad776433fe3a1cf6854
Author: Chris Cormack <chris@bigballofwax.co.nz>
Date: Wed Nov 9 08:28:26 2011 +1300
Starting work on webservices
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Patch includes database update adding 2 new fields to the saved_sql table.
1) Checked that adding/deleting/editing of sql reports still works and new
fields are correctly saved to the database.
2) The saved reports page now lists the new fields correctly.
3) Checking URLs with JSON output
- if report is not public http://localhost/cgi-bin/koha/svc/report?id=2 fails,
but http://localhost:8080/cgi-bin/koha/svc/report?id=2 works
- if report is public, both links work
4) Checking results are cached
Created a report listing my borrowers, setting caching to 60 seconds and
added new borrowers before doing following tests.
- URL doesn't show new borrower immediately - correct
- Turning off the usecache system preference updates data - correct
- Restarting memcached updates data '/etc/init.d/memcached restart' - correct
- Waiting until cache time runs out updates data - correct
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Retested the feature and renewing my sign-off.
There are some things that should be noted:
- currently there is a hardcoded limit of 10 results for the reports webservice
- reports with runtime parameters are currently not supported
