Commit graph

565 commits

Author SHA1 Message Date
3e164d200a
Bug 36100: (QA follow-up) Move 400 to 409
Duplicate exceptions tend to generate a 409 in our REST API.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:42 +01:00
e8a2f4df73
Bug 36100: Remove readOnly from bookings definition.
Whilst readOnly feels correct here, swagger doesn't work that way at our
version. It must be removed to restore the ability to edit exising
bookings.

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-03-07 16:36:41 +01:00
Katrin Fischer
dbc67ce351
Bug 35368: Add public to "Add a new checkout" in API documentation
Without this patch the "Add a new checkout" label showed
up twice in the Checkouts section of the API documentation.
This adds (public) to the end of one, to make it possible
to distinguish them.

To test (ON YOUR HOST MACHINE):
1. Run:
   $ cd api/v1/swagger
   $ docker run --rm -v $(pwd):/api --workdir /api redocly/cli \
           build-docs swagger.yaml --output index.html
2. Open the generated index.html in your browser
3. Verify doubled up headings
4. Apply patch
5. Re-run docker command form 1.
6. Verify there are now 2 different labels and "(public)" is on the
   right one.

Note: if you have wrong permissions on the file, chown it to your
own user to open it in the browser.

Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-30 17:48:40 +01:00
cd9a3c3a07
Bug 30230: (QA follow-up) Also add list_borrowers to the singular endpoint
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-29 12:25:11 +01:00
Thibaud Guillot
fc75a344b8
Bug 30230: Add new 'list_borrowers' permission
When a patron search is performed only a user with edit_borrowers
permission can search by name. Search can works only with cardnumber but
it makes searching less intuitive I think.

So, as mentioned in the discussion, I've added a new 'list_borrowers' permission,
completely independent of 'edit_borrowers', so that I can search for a member via the interface
and get the results. In addition to the permission to perform check in and checkouts, this no longer poses an obstacle to simple use.

Test plan:

1) Check with a user without 'edit_borrowers' permission that the patron search can only be performed with cardnumber
2) Apply this patch
3) Make the updatedatabase to add new 'list_borrowers' permissions
4) Set 'list_borrowers' permission on one user and see the difference

Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-29 12:25:08 +01:00
5464d11099
Bug 35744: Implement +strings for GET /patrons/:patron_id
In order to retrieve the library's name and patron category's
description alongwith other patron's info.

Test plan:
Run the following command before and after this patch:

% curl -u koha:koha --request GET 'http://localhost:8081/api/v1/patrons/42' --header "Content-Type: application/json" --header "x-koha-embed: +strings" | jq

Notice that you now have _strings which contains the library's name and
patron category's description

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-29 12:24:59 +01:00
8b44fb4d18
Bug 30070: EDIFACT and ID are abbreviations
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-17 11:38:59 +01:00
8b5a5f46cb
Bug 30070: (follow-up) Filter out deleted files on the API
EDI uses a 'deleted' flag in the table to denote deleted files, we
should filter those out of the API response.

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-17 11:38:57 +01:00
9382571968
Bug 30070: Add fetch endpoint for EDI Files
This patch adds a simple EDI Files listing REST endpoint

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-17 11:38:53 +01:00
2fbd47fdd6
Bug 35634: (bug 33105 follow-up) fix typo issues_manage vs issue_manage
issue_manage inserted into the DB but issues_manage is tested in the
controller script.

Test plan:
  git grep issues_manage
should no longer return any results

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-11 15:44:46 +01:00
520ac94242
Bug 35757: Add new fields to api definitions
The counter registry API has added a new field to both the counter registry and sushi service endpoints. This patch adds those new fields to the definitions.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-11 12:10:44 +01:00
54b07ffc18
Bug 35658: Fix misaligned collectionFormat in patron_holds.yaml
This patch fixes a silly typo.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-01-04 16:51:43 +01:00
73eb73e3e6
Bug 35574: Correct api permission for listing biblio specific bookings
To test:
1 - Follow above plan and apply only first patch
2 - Not the table does not load (403: Forbidden)
3 - Apply this patch
4 - Reload
5 - Success!

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2023-12-18 15:21:05 +01:00
8ed228ed9e
Bug 35468: Unit tests
This patch adds all the 'basic' unit tests one would expect to find with
the api endpoints.  I've been working on this as a follow-up to bug
29002 as promised, but hadn't yet submitted them as I wanted to get the
full coverage.. but having these basics in early makes sense whilst I
continue on the more advanced cases.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2023-12-14 16:29:18 +01:00
21f341c1e0
Bug 35468: Correct api bookings permission
At some point during development we migrated from 'parameters' to
'circulate' for the manage_bookings sub-permission. Unfortunately, the
corresponsing API definition missed the update.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2023-12-14 16:29:18 +01:00
a5f3815c65
Bug 34287: Add check on public availability endpoint
A quick check for patron equals current user in the public availability
endpoint.

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-24 16:43:22 -03:00
a811dc236b
Bug 35219: Fix ERM usage endpoints to show in docs
This patch fixes the tags and also adds the tags to the swagger.yaml
file to allow the endpoints to be documented correctly.

One endpoint has also been deleted as it is no longer required.

Test plan:
Check the attached files to see that all tags are now prefixed with
'erm_' and that the swagger file now includes an entry for all of these
files

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-06 08:42:40 -03:00
05c28e5e1c
Bug 35230: (bug 30708 follow-up) fix description for catalogue_item
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:47 -03:00
19a4e0f492
Bug 29002: (QA follow-up) Fix stray x-koha-embed
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:31 -03:00
e6224e960f
Bug 29002: (follow-up) Rebase fix, remove q_header
We have removed q_header support in master, so we now need to remove it
in this patchset too.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:15 -03:00
2472214301
Bug 29002: Add bookable state to items
This patch adds a bookable boolean to enable/disable the ability to book
an item ahead of time

Test plan
1) Navigate to the 'Items' tab of a biblio
2) Note the new 'Bookable' option and select at least one item to allow
   bookings to take place
3) Note that without any items selected as 'bookable' one does not have
   the 'Place booking' option or the 'Bookings' tab on the biblio
   details page.
4) Note that when at least one item is bookable, the place booking modal
   now only displays items that are marked as bookable in the item
   selection
5) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:11 -03:00
902d7e4fb0
Bug 29002: Add bookings view
This patch adds a new view to the biblio details page to display any
bookings that may have been made.

Test plan
1) Run through the steps from the previous patch
2) Note the appearance of a new 'Bookings' option in the left menu
3) Click the bookings option to load the new bookings page for the
   biblio
4) See that the booking from the previous patch appears on the page
5) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:08 -03:00
9fff2e1f4e
Bug 29002: Add ability to book material from the staff client
This patch introduces a new modal to the biblio details page to allow
booking of materials.

Test plan
1) Navigate to the details page of a biblio
2) Note the new 'Place booking' button in the toolbar
3) Click the new button and note the new modal dialogue
4) Enter part of a patron name or cardnumber and then select from the
   presented results
5) Optionally pick an item from the select list
6) Select a start date and end date from the calender
7) Submit
8) Attempt to book the same item to another user, note that the dates
   previously selected are now greyed out.
9) Experiment with different items and all items options to confirm the
   available slots in the datepicker update as expected.
10) Sign off

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:07 -03:00
e8d5219547
Bug 29002: Add bookings objects and API classes
This patch adds new Koha::Object based classes for bookings logic and
adds API controllers to expose the new bookings data via the REST API's.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Janet McGowan <janet.mcgowan@ptfs-europe.com>
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Laurence Rault <laurence.rault@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:04:07 -03:00
78589dbe6e
Bug 34008: Harmonize attribute names
This patch harmonizes the attribute names with what is used for `items`
and `checkouts` in terms of terminology.

It also adapts the tests so they are less random failure-prone (they had
a fixed value for the item type, which might make things explode if the
chosen value already exists on the DB.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:15 -03:00
05f0604d60
Bug 34008: (QA follow-up) 'item_type_id' should be used
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:15 -03:00
Andreas Jonsson
82bdf93ab7
Bug 34008: Add REST endpoint for list of itemtypes
Test plan:
* Enable the system preference RESTBasicAuth
* curl -s --request GET http://kohadev-intra.mydnsname.org:8081/api/v1/itemtypes
  should give 401 Unauthorized
* curl -s -u koha:koha --request GET http://kohadev-intra.mydnsname.org:8081/api/v1/itemtypes
  should produce JSON-list of itemtypes
* curl -s -u koha:koha --header "x-koha-embed: translated_descriptions" --request GET http://kohadev-intra.mydnsname.org:8081/api/v1/itemtypes
  should include the field translated_descriptions containing the translated descriptions, if any

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] perltidy -b t/db_dependent/api/v1/itemtypes.t # Resolve bad score of 44
[EDIT] chmod 755 t/db_dependent/api/v1/itemtypes.t
[EDIT] perltidy -b Koha/REST/V1/ItemTypes.pm
Lesson: Please run qa tools yourself and adjust accordingly?
Edit (tcohen): I restored the item_type_translated_description.yaml file
as the entire API was broken because of the lack of it.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:14 -03:00
Magnus Enger
6fec4ea0df
Bug 26170: Add protected status for patrons
This set of patches makes it possible to protect patrons from being accidetally
deleted or merged with other patrons, from the UI and from (well behaved) cron
jobs. The following subroutines are affected:
- Koha::Patron::delete
- Koha::Patron::merge_with
- Koha::Patron::safe_to_delete
- C4::Members::GetBorrowersToExpunge

Please note:
- This does not intend to protect patrons from being edited, only from being
  deleted

To test:

* Tests
- Run the affected tests:
  prove t/db_dependent/Members.t
  prove t/db_dependent/Koha/Patrons.t

* Editing protected status and manual deletion
- Add a new user, note the presence of the "Protected" field under "Library
  management", but leave it at the default "No", for now.
- Note that "Protected" is displayed in the "Library use" section of the patron
  details.
- Note that More > Delete is avaiable as an action when the patron is saved
- Edit the user and set "Protected" to "Yes"
- Note that More > Delete is now disabled, with a note that the patron is protected

* Batch patron deletion
- Go to Tools > Batch patron deletion and anonymization
- Check the box for "Verify you want to delete patrons"
- Choose the category of your protected patron for "whose patron category is"
  and click "Next" to run the actual deletion
- Check that your protected patron was not deleted

* Merging patrons
- Make sure you have two patrons with similar names or the same category, so
  you can find them with one search. One should be protected, one not.
- Search for the patrons, tick their boxes and click on "Merge selected patrons"
- Select one of the patrons as the "patron to keep".
. Click on "Merge patrons"
- "No valid patrons to merge were found" should be shown
- Repeat this with the other patron as the "patron to keep"
(A future enhancement could be to not allow a protected patron to be selected for
merging in the first place.)

* misc/cronjobs/delete_patrons.pl
- Make sure you have a protected patron, in a category with at least one more
  patron.
- Run something like this (at least in ktd):
  $ perl misc/cronjobs/delete_patrons.pl --category_code <code> -v --confirm
  (Replace <code> with the actual categorycode.)
- Make sure the borrowernumber of the protected patron is not mentioned in the
  output of the script.
- Check the protected patron was not deleted
- Check the non-protected patrons were deleted

* REST API (with ktd)
- Make sure you still have a protected patron, and note their borrowernumber
- Enable RESTBasicAuth and restart all the things
- Run these two commands from the command line on the host:
  $ curl -u koha:koha --request GET "http://localhost:8081/api/v1/patrons/54"
  $ curl -u koha:koha --request DELETE "http://localhost:8081/api/v1/patrons/54"
  (Replace 54 with the actual borrowernumber of your protected patron.)
- The first curl command should give you the patron details. The second should
  give this output:
  {"error":"Protected patrons cannot be deleted","error_code":"is_protected"}

There could be more functions/scripts where patrons are deleted that I have not
thought about. Please report them on the bug if you find any!

Update 2023-10-19: Fix "More > Delete" on patron, so link can not be clicked.
Update 2023-10-19: Rebase

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 17:23:06 -03:00
3ff7771d94
Bug 34587: Fix stray type definition
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-01 15:00:17 -03:00
38f8e116fa
Bug 34587: Fix ->validation->output in API endpoints
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:30 -03:00
f7e2bc5d9b
Bug 34587: Move YOP to mus/yus instead of erm_usage_titles
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:29 -03:00
184602ce38
Bug 34587: Add proprietary_id and platform to erm_usage_titles
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:26 -03:00
d55ea648ac
Bug 34587: Improve SUSHI COUNTER error handling
This is doing a few things, checking for sushi errors:
1) Abort background job and provide error message if SUSHI returned
   {Severity} in response, this means SUSHI error, docs here:
https://app.swaggerhub.com/apis/COUNTER/counter-sushi_5_0_api/5.0.2#/SUSHI_error_model

2) Abort background job and provide error message if SUSHI provided
   proper response, but contains Exceptions in Report_Header

3) Abort background job and provide error message if SUSHI returned 0
   report items (e.g. everything is fine but there are no items for the
   harvest period provided

4) Abort background job and provide error message if COUNTER file
   created is larger than allowed by the database.

5) Provide error message if manual upload file size exceeds the max
   allowed by the database

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:25 -03:00
979ff589d9
Bug 34587: Remove counter files from API embedding to improve performance
Rather than fetching the counter files and embedding the counter logs, we now add a foreign key to the data provider in the counter logs table and fetch them directly.

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:21 -03:00
ca8ea02fda
Bug 34587: Move everything into a ERM/EUsage subfolder
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:18 -03:00
b7aebb337e
Bug 34587: Add integration with the counter registry API
The counter registry offers an API that provides SUSHI information for each provider. This will be incredibly useful for creating new providers as we can use the correct information from the registry for harvesting urls etc. This reduces the risk of user input errors when creating providers and gives greater reliability to the data required to successfully harvest from the SUSHI API

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:17 -03:00
dd37e0075a
Bug 34587: Remove q_header parameter from API specs
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:16 -03:00
cf87082682
Bug 34587: File upload - Background job
ErmSushiHarvester background job will now be initiated by either
enqueue_counter_file_processing_job or by enqueue_sushi_harvest_jobs
from Koha/ERM/UsageDataProvider.pm, the former if triggered by a manual
file upload, the latter if by the 'run now' button or by the cron script.

This commit also includes some rewording/refactoring, namely:
- COUNTER file validation now happens in the API, before enqueuing the job.
- Removal of no longer used POST /erm/counter_files endpoint
- Koha/ERM/UsageDataProvider.pm:
-- run method is now enqueue_sushi_harvest_jobs
-- new enqueue_counter_file_processing_job method
-- harvest method is now harvest_sushi
-- new set_background_job_callbacks method to set the background job callbacks
- REST/V1/ERM/UsageDataProviders
-- run method is now process_SUSHI_response
-- new process_COUNTER_file endpoint

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:15 -03:00
9fc2151ead
Bug 34587: Remove begin_date and end_date as usage_data_providers properties/columns
(cherry picked from commit 9339eed9358301f7bf17934e2e13fc17205d9cd0)

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:14 -03:00
16592d267a
Bug 34587: Accept date inputs upon confirming 'run now' button
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:13 -03:00
ffa16245d3
Bug 34587: Make start and end dates in providers summary data dependent
Currently the start and end dates in the summary tab are based on the earliest and latest harvest run, rather than the earliest and latest data harvested. This should be changed so that we can see the period of data harvested for each provider

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:12 -03:00
b74815dc64
Bug 34587: Add list endpoints for usage_databases/items/platforms
Needed to display lists in the provider tabs in the UI

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:08 -03:00
8987ff1591
Bug 34587: Abstract reports backend to allow new data types
Now that harvesting is possible for platforms, databases and items we need to be able to generate reports for all of these data types. Currently the reporting backend structure is very geared towards titles. Rather than copying this for each different data type, this patch abstracts the code to accept the data type as a url parameter and use that to generate a report based on a given data type

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:07 -03:00
e1583a334a
Bug 34587: Add reports by data provider
Add the option to have a report by provider that rolls all usage up into one top-level figure to see how often that provider is being used a given period

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:03 -03:00
ffbcf83b95
Bug 34587: Add default report capability to reports viewer
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:02 -03:00
8cc8b2501e
Bug 34587: Add reports by metric types
Adds the ability to generate a report with the individual metric types as columns

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:45:59 -03:00
c1455422e4
Bug 34587: Add test connection button
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:45:59 -03:00
2951edc697
Bug 34587: SUSHI COUNTER Harvester
This commit is a squash of the following:

SUSHI harvesting process in the data providers class:
* Builds the URL query and requests the SUSHI service endpoint
* Parses the JSON response and builds the csv COUNTER file and adds it to counter_files table

Usage statistics data processing:
* When a counter_files entry is stored, CounterFile.pm will:
* Parse the csv COUNTER file and
* Add a usage_titles entry for each unique title in the COUNTER file
* Add the title's respective erm_usage_mus (monthly usage) entries, repeating for each metric_type
* Add the title's respective erm_usage_yus (yearly usage) entries, repeating for each metric_type

Harvesting cronjob;

'Run now':
* API endpoint to start the harvesting process of a data provider
* Button in the data providers list to run the harvesting process for each data provider upon clicked

ERM SUSHI: Background job

Job progress is updated to total amount of usage titles after retrieving
the response from SUSHI;
Job warning and success messages are added accordingly
Redundant duplicate titles will not be added
Redundant duplicate monthly and yearly usage statistics will not be added
Data provider harvest background job harvests once per report_type

Enqueue one background job for each report_type in the usage data provider

Update the way we measure progress in the background job.
It now uses the COUNTER report body rows instead of SUSHI response results.
We're now incrementing and showing the number of skipped mus, skipped
yus, added mus and added yus
There's a bug in the way we calculate yus
Updates to background job progress bar - Depends on 34468

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:45:58 -03:00
5125195a52
Bug 34587: API swagger paths and definitions
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:45:53 -03:00
c0eb930e98
Bug 29523: (follow-up) Adding documentation to swagger.yaml
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-27 16:40:47 -03:00