The counter registry offers an API that provides SUSHI information for each provider. This will be incredibly useful for creating new providers as we can use the correct information from the registry for harvesting urls etc. This reduces the risk of user input errors when creating providers and gives greater reliability to the data required to successfully harvest from the SUSHI API
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
ErmSushiHarvester background job will now be initiated by either
enqueue_counter_file_processing_job or by enqueue_sushi_harvest_jobs
from Koha/ERM/UsageDataProvider.pm, the former if triggered by a manual
file upload, the latter if by the 'run now' button or by the cron script.
This commit also includes some rewording/refactoring, namely:
- COUNTER file validation now happens in the API, before enqueuing the job.
- Removal of no longer used POST /erm/counter_files endpoint
- Koha/ERM/UsageDataProvider.pm:
-- run method is now enqueue_sushi_harvest_jobs
-- new enqueue_counter_file_processing_job method
-- harvest method is now harvest_sushi
-- new set_background_job_callbacks method to set the background job callbacks
- REST/V1/ERM/UsageDataProviders
-- run method is now process_SUSHI_response
-- new process_COUNTER_file endpoint
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
(cherry picked from commit 9339eed9358301f7bf17934e2e13fc17205d9cd0)
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Currently the start and end dates in the summary tab are based on the earliest and latest harvest run, rather than the earliest and latest data harvested. This should be changed so that we can see the period of data harvested for each provider
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Needed to display lists in the provider tabs in the UI
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Now that harvesting is possible for platforms, databases and items we need to be able to generate reports for all of these data types. Currently the reporting backend structure is very geared towards titles. Rather than copying this for each different data type, this patch abstracts the code to accept the data type as a url parameter and use that to generate a report based on a given data type
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Add the option to have a report by provider that rolls all usage up into one top-level figure to see how often that provider is being used a given period
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Adds the ability to generate a report with the individual metric types as columns
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This commit is a squash of the following:
SUSHI harvesting process in the data providers class:
* Builds the URL query and requests the SUSHI service endpoint
* Parses the JSON response and builds the csv COUNTER file and adds it to counter_files table
Usage statistics data processing:
* When a counter_files entry is stored, CounterFile.pm will:
* Parse the csv COUNTER file and
* Add a usage_titles entry for each unique title in the COUNTER file
* Add the title's respective erm_usage_mus (monthly usage) entries, repeating for each metric_type
* Add the title's respective erm_usage_yus (yearly usage) entries, repeating for each metric_type
Harvesting cronjob;
'Run now':
* API endpoint to start the harvesting process of a data provider
* Button in the data providers list to run the harvesting process for each data provider upon clicked
ERM SUSHI: Background job
Job progress is updated to total amount of usage titles after retrieving
the response from SUSHI;
Job warning and success messages are added accordingly
Redundant duplicate titles will not be added
Redundant duplicate monthly and yearly usage statistics will not be added
Data provider harvest background job harvests once per report_type
Enqueue one background job for each report_type in the usage data provider
Update the way we measure progress in the background job.
It now uses the COUNTER report body rows instead of SUSHI response results.
We're now incrementing and showing the number of skipped mus, skipped
yus, added mus and added yus
There's a bug in the way we calculate yus
Updates to background job progress bar - Depends on 34468
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 32721: (QA follow-up) Rename fields to opac*
This patch updates the field names to reflect that they're OPAC
related.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 32721: (QA follow-up) Fix rebase errors
We let some superflous template params creep back in during a rebase
somewhere.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds the columns for userjs and usercss to the branches table
Test plan as per previous commit
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
In case the logged in user does not have manage_sysprefs we should no
display the form in the settings.
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
To retrieve the sysprefs, instead of using the svc script. See bug
33606.
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
It broke the REST API routes.
Adjustement for bug 33971.
Signed-off-by: Heather Hernandez <heather_hernandez@nps.gov>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Update statuscode -> status_code on the js files
Update remaining batch_id -> ill_batch_id
Update batch object in Illrequest.pm strings_map
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch takes on normalizing the attribute names, embeds, and also
makes the whole API more kosher, in terms of using accessors for related
objects, using the standard structure for strings_map, etc.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Update accessors
Add +strings embed
Add x-koha-embed to batches list andpoint
Add embed to API call from the front-end
Update table to get data from _strings
Add x-koha-embed to tests
Add strings_map to Illbatch
Add to_api_mapping to Illbatch
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
- UI adding support for batch statuses in batch UI
- Admin UI for managing batch statuses
- API specs
Co-authored-by: Andrew Isherwood <andrew.isherwood@ptfs-europe.com>
Signed-off-by: Edith Speller <Edith.Speller@ukhsa.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
- Add batch column to requests table
- Establish if there are any availability or metadata enrichment plugins and pass that to the template
- Verify if we have any backend that can support batches, if not, don't show the option
- Updates to the ILL toolbar
- New ILL batch modal
- New Koha classes
- API specs
Co-authored-by: Andrew Isherwood <andrew.isherwood@ptfs-europe.com>
Signed-off-by: Edith Speller <Edith.Speller@ukhsa.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
- Adds 'batch' accessor to Illrequest object
- New illbatches and illbatch_statuses tables
- New foreign key 'batch_id' in illrequests table
- Atomic update file
- Default illbatch_statuses
- Add 'add' ill_requests api method
- Add POST method in ill_requests path
- Add 'batch_id property to ill_request api definition
- Updated swagger.yml with new batches and batchstatuses endpoints
Co-authored-by: Andrew Isherwood <andrew.isherwood@ptfs-europe.com>
Signed-off-by: Edith Speller <Edith.Speller@ukhsa.gov.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch converts the existing work to use x-koha-override instead of
introducing an entirely new header and we default to the system
preference value if the override isn't passed.
I believe this is more consistent with our approach in other areas of
koha to date.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
It would be nice to have the ability to send a welcome notice when creating patrons via the API.
Test Plan:
1) Apply this patch
2) Ensure you have a WELCOME notice
3) Create a new patron using the REST API ( api/v1/patrons )
4) Note no welcome notice is sent to the patron ( you can check the
notices tab for the patron )
5) Repeat step 3, but send the header X-Koha-SendWelcomeEmail with a
value of 1 as part of the POST
6) Note the welcome message for the patron is in their notices!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 33690: Tidy test
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Test plan:
Request the /items REST API endpoint and ask for the biblio's info to be
embeded into the response passing x-koha-embed
curl -u koha:koha --request GET 'http://localhost:8081/api/v1/items' --header "x-koha-embed: biblio" --header "Content-Type: application/json"
Sponsored-by: BULAC - http://www.bulac.fr/
Signed-off-by: Andreas Roussos <a.roussos@dataly.gr>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds the `cancellation_requested` attribute to the hold
object definition, and allows embeding it as on the different holds
endpoints that migt be useful.
To test:
1. Apply this patches
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/Hold.t \
t/db_dependent/api/v1/*holds.t
=> SUCCESS: Tests pass!
3. Play with your REST tool (Postman?) on the following endpoints:
GET http://localhost:8081/api/v1/holds
GET http://localhost:8081/api/v1/patrons/:patron_id/holds
on both, pass and not pass the `x-koha-embed` header with
`cancellation_requested` on it.
=> SUCCESS: It is easy! You see the attribute and you don't, and the
content makes sense!
4. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The API route for listing all suggestion:
/api/v1/suggestions
send back an error message when there is a suugestion with non standard
status (ASKED, CHECKED, ACCEPTED, REJECTED).
This patch fixes this too restrictive restriction.
TO TEST:
1. Add a status in SUGGEST_STATUS AV list.
2. Create a suggestion, and assign it to the previsous status.
3. Call /api/v1/suggestion
3. You get an error message:
{
"errors": [ {
"message":"Not in enum list: ASKED, CHECKED, ACCEPTED, REJECTED.",
"path":"\/1\/status"
}],
"status":200
}
4. Apply the patch. Call /api/v1/suggestion
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Currently the erm_eholdings_titles table has a field called preceeding_publication_title_id. This should be preceding_publication_title_id
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch makes the password validation response return the following
patron attributes to the API consumer:
* cardnumber
* userid
This will give hints on what was used to validate in the fallback
bahvior the endpoint has.
To test:
1. Apply the unit tests patch
2. Run:
$ ktd --shell
k$ prove t/db_dependent/api/v1/password_validation.t
=> FAIL: The endpoint doesn't return this valuable data
3. Apply this patch
4. Repeat 2
=> SUCESS: Tests pass! We got the cardnumber and the userid!
5. Sign off :-D
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
