Commit graph

29 commits

Author SHA1 Message Date
f1f9c6dc74 Bug 26384: Fix executable flags
.pm must not have -x
.t must have -x
.pl must have -x

Test plan:
Apply only the first patch, run the tests and confirm that the failures
make sense
Apply this patch and confirm that the test now returns green

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-09-11 09:56:56 +02:00
c66637670b
Bug 21684: Fix UploadedFile[s]->delete
Tests were failing with:
 #   Failed test 'Test delete via UploadedFile as well as UploadedFiles'
 #   at t/db_dependent/Upload.t line 193.
 DBIx::Class::Row::delete(): Not in database at /kohadevbox/koha/Koha/Object.pm line 219

I am not sure this patch is perfect, a set of uploaded files should be
deleted in a transaction, which would be rollback if something is wrong.
But it will be tricky to restore the files after they have been deleted.
It seems that we should deal with that with a more complicated process
and should be part of a separate bug.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-01-07 16:32:53 +00:00
15fd90dcbc Bug 19633: Use alphanumeric error codes in upload
The error codes 1 to 7 are used in Uploader.pm or tools/upload.pl.
It would be nice to use alphanumeric codes instead.
No behavior change expected.

Test plan:
[1] Run t/db_dependent/Upload.t
[2] Verify that a regular upload with tools/upload.pl still works.
[3] Rename upload_path in your koha-conf.xml. Restart Plack, flush the cache
    and try to upload to a category. Correct error message?
[4] Upload the same file twice to the same category.
    Correct error message the second time?

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Bug 19633: (QA follow-up) Really remove these ugly numbers

See BZ comment5. We now remove the numbers also from the constant names.

Test plan:
Read the changes.
Git grep "ERRCODE_"
Run t/db_dependent/Upload.t (Note: You may see one failure here; it is fixed
on bug 20727. So depends on who reaches master first.)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-08-10 10:10:46 +00:00
f63bb8b0ca Bug 20727: (QA follow-up) Fix Upload.t and Plugins.t
Trivial changes.
Adding C4::Context->temporary_directory to subsequent tmpdir call in Plugins.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-06-22 16:10:10 +00:00
Kyle M Hall
3ff7c27d98 Bug 20727: Move temporary_directory() to C4::Context
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-06-22 16:10:09 +00:00
6d150eaeca Bug 20081: Set inline headers for uploaded pdfs
To test:
1 - Edit a framework and add the 'upload/pl' plugin to the 856 field
2 - Attach a pdf to one record
3 - Attach a different type of file to another
(To attach, edit a record in the framework edited above and find the
 helper by the 856 field)
4 - Click the links on the record
5 - Verify the pdf opens inline
6 - Verify the other file does not
7 - prove t/db_dependent/Upload.t

Signed-off-by: Claire Gravely <claire.gravely@bsz-bw.de>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2018-03-26 17:31:17 -03:00
b25cb7ab0f Bug 19420: (QA Follow-up) Fix test, correct POD
The test needs the code level in the error hash now.
And POD is adjusted accordingly.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-11-08 13:31:37 -03:00
4eb53eeee9 Bug 18300: [QA Follow-up] Fix return value inconsistency
As noted on bug report 17669, the return values of delete (both singular
and plural), delete_missing and delete_temporary should be consistent.

Removed the if-construction around full_path. We do not need it; in the
very exceptional case that full_path would be empty, we should delete
the record since the file is missing.

Adjusted POD and unit tests accordingly.

Test plan:
Run t/db_dependent/Upload.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-21 00:11:40 +00:00
8c80c9b8e1 Bug 18300: Delete missing upload records
If you deleted files from the upload directories manually, or you rebooted
with files in the temporary upload folder, or for some other reason have
records without a file, you may want to cleanup your records in the
uploaded_files table.

This patch adds the method delete_missing to Koha::UploadedFiles. It also
supports a keep_record parameter to do a dry run (count the missing files
first).

Also, we add an option --uploads-missing to cleanup_database. If you add
the flag 1 after this option, you will delete missing files. If you add the
flag 0 or only use the option, you will count missing files.

A subtest is added to Upload.t for delete_missing tests.

Test plan:
[1] Run t/db_dependent/Upload.t
[2] Upload a file and delete the file manually.
[3] Run cleanup_database.pl --uploads-missing
    It should report at least one missing file now.
    Check that the record has not been deleted.
[4] Run cleanup_database.pl --uploads-missing 1
    It should report that it removed at least one file.
    Check that the record is gone.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-21 00:11:39 +00:00
8e99f884cc Bug 17669: Remove warning 'variable $kohaobj masks earlier declaration in same scope'
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-21 13:55:25 -04:00
aee0682d60 Bug 17669: [QA Follow-up] More consistency in return values of delete
See Bugzilla comment36 (QA request).

Koha::UploadedFile->delete
Returns 1, 0E0 or -1 (unknown).

Koha::UploadedFiles->delete and delete_temporary
Returns number of deleted records, 0E0 or -1.

POD lines are corrected accordingly. Unit tests adjusted too.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Added a note that Koha::Object->delete itself is not completely consistent
with Koha::Objects->delete (DBIx). The singular may return 1 when it
gets 0E0 from DBIx, while the plural one may return 0E0. But should be
handled somewhere else.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-21 13:55:25 -04:00
05e25dbc18 Bug 17669: [QA Follow-up] Allow zero in temp-uploads-days
As requested by QA on comment33.
If the pref is 0 or the overriding command line parameter is 0, all
temporary files will be deleted. But if the pref is NULL or empty string,
we will not delete files.

Also adjusted the description of the preference in this regard.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-21 13:55:25 -04:00
bab3a204be Bug 17669: [QA Follow-up] Rename preference by removing underscores
Requested by QA on comment25.
Result of:
git grep -l Upload_PurgeTemporaryFiles_Days | xargs sed -i -e "s/Upload_PurgeTemporaryFiles_Days/UploadPurgeTemporaryFilesDays/g"

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-21 13:55:25 -04:00
53f5d55ab4 Bug 17669: Add delete_temporary method with unit tests
Test plan:
Run t/db_dependent/Upload.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-21 13:55:24 -04:00
e429db4746 Bug 17501: [Follow-up] QA Requests
This patch makes the following changes, as requested by QA:

[1] UploadedFile->delete always calls SUPER::delete.
    The return value normally comes from SUPER::delete; if removing the
    file failed, we return false. Two warns are kept.
    Since delete does no longer return the filename, a few changes were
    needed in tools/upload.pl.
[2] Method getCategories is moved to UploadedFiles. Script tools/upload.pl
    now only contains one call. Added a use C4::Koha.
[3] Calls UploadedFiles->delete as class method. As a result I removed
    method delete_errors for now; may be reconsidered on a new report.
[4] Adjusted three ->search calls for id and public to ->find calls.
[5] If you pass no id to upload.pl when deleting, you don't get an alert.

All by all, we got rid of 15 lines !

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 14:20:07 +00:00
a118dd2ba0 Bug 17501: Additional polishing (POD, unit tests)
This patch adds some documentation lines.
And mainly rearrangs the tests in Upload.t. The 'basic CRUD testing' is
not needed separately any more. A new test catches the "file missing"
warn.

Test plan:
[1] Run perldoc on UploadedFile[s].pm
[2] Run t/db_dependent/Upload.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 14:20:06 +00:00
3acee79310 Bug 17501: Rename Upload to Uploader
Why? Koha::Uploader now only contains the actual CGI upload. The new name
better reflects its handler status.
Pragmatically, the difference between Uploaded and Uploader makes it
easier to specifically search for them in the codebase.

Test plan:
[1] Run t/db_dependent/Upload.t.
[2] Add an upload via the interface.
[3] Check the code:
    git grep "Koha::Upload;"
    git grep "Koha::Upload\->"

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 14:20:06 +00:00
ba5cd24553 Bug 17501: Move getCategories and httpheaders from Upload.pm
Class method getCategories has no strict binding to Upload.pm. While
Upload.pm is now restricted to the actual uploading process with CGI
hook, this routine fits better in the UploadedFile package.

Class method httpheaders can be moved as well for the same reason. Note
that it actually is an instance method. The parameter $name is dropped.

Test plan:
[1] Run t/db_dependent/Upload.t.
[2] Check the categories in the combo box of tools/upload.
[3] Check a download via tools/upload and opac-retrieve-file.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 14:20:05 +00:00
158442eb9e Bug 17501: Remove Koha::Upload::get from Koha::Upload
The get routine actually returns records from uploaded_files. It should be
possible to replace its calls by direct calls of Koha::UploadedFiles.

This patch is the crux of this patch set. It deals with all scripts that
use Koha::Upload.

In the process we do:
[1] Add a file_handle method to Koha::UploadedFile. This was previously
    arranged via the fh parameter of get.
[2] Add a full_path method to UploadedFile. Previously returned in the
    path hash key of get. (Name is replaced by filename.)
[3] Add a search_term method too (implementing get({ term => .. }).
    This logic came from _lookup.
[4] Add a keep_file parameter to delete method. Only used in test now.

Test plan:
[1] Run t/db_dependent/Upload.t
[2] Go to Tools/Upload. Add an upload, download and delete.
[3] Add another public upload , search for it.
    Use the hashvalue to download via opac with URL:
        cgi-bin/koha/opac-retrieve-file.pl?id=[hashvalue]
[4] Go to Tools/Stage MARC for import. Import a marc file.
[5] Go to Tools/Upload local cover image. Import an image file.
    Enable OPACLocalCoverImages to see result.
[6] Test uploading a offline circulation file:
    Enable AllowOfflineCirculation, and create a koc file (plain text):
    Line1: Version=1.0\tA=1\tB=2
    Line2: 2016-11-23 16:00:00 345\treturn\t[barcode]
    Note: Replace tabs and barcode. The number of tabs is essential!
    Checkout the item with your barcode.
    Go to Circulation/Offline circulation file upload.
    Upload and click Apply directly.
    Checkout again. Repeat Offline circulation file upload.
    Now click Add to offline circulation queue.
[7] Connect the upload plugin to field 856$u.
    Enable HTML5MediaEnabled.
    Upload a webm file via the plugin. Click Choose to save the URL,
    and put 'video/webm' into 856$q. Save the biblio record.
    Check if you see the media tab with player on staff detail.
    (See also: Bug 17673 about empty OPACBaseURL.)

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 14:20:05 +00:00
7a62005468 Bug 17501: Move Koha::Upload::delete to Koha::UploadedFile[s]
Since delete is not part of the upload process, we will move it now
to Koha::UploadedFile[s].
Deleting the file will be done in UploadedFile.
The (multiple) delete method in UploadedFiles refers to the single delete.

Test plan:
[1] Run t/db_dependent/Upload.t
    The warning ("but file was missing") in the last subtest is fine;
    the file did not exist. Will be addressed in a follow-up.
[2] Search for uploads on Tools/Upload. Clone this tab (repeat search on
    a new tab in your browser).
[3] Delete an existing upload on the first tab.
[4] Try to delete it again on the second tab. Error message?
[5] Bonus points:
    Add an upload. Mark the file immutable with chattr +i. Try to delete
    the file. You should see a "Could not be deleted"-message.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 14:20:04 +00:00
44d4dc7040 Bug 17501: Introduce Koha::Object[s] classes for UploadedFile(s)
In the next set of patches we will start using these new classes in
Koha::Upload, and scripts using it.
This is just the starting point of that migration.

Test plan:
[1] Run t/db_dependent/Upload.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-01-20 14:20:03 +00:00
be9a609870 Bug 17538: (Bug 17216 follow-up) Fix t/db_dependent/Upload.t
The insert into authorised_values failed because of the missing FK

Test plan:
  prove t/db_dependent/Upload.t
should return green

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-11-04 11:04:51 +00:00
e8117033e4 Bug 16501: Remove some unneeded warns in Upload.t
At some point I saw some gain in warning about a missing config line.
But these warns are actually not needed here.
Removing them here.

Test plan:
Run t/db_dependent/Upload.t while having upload_path in your koha-conf and
also without it.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-05-16 17:46:25 +00:00
2c385d0d7d Bug 14686: Remove unused use
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
2016-04-27 16:14:18 +00:00
220a517c8e Bug 14686: [QA Follow-up] Move allows_add_by to Upload.pm
As requested by QA, the sub should be moved.

Test plan:
See previous patches.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
2016-04-27 16:14:18 +00:00
b69402d14c Bug 14686: Add allows_add_by to check permissions for uploading
The three permissions in tools/upload-file.pl are moved to a custom
routine in DBIx's UploadedFile.pm. An additional granular permission
upload_general_files is added. (The dbrev patch will contain it too.)
At some point in time this could be moved to a Koha::Object class.

The routine is tested in Upload.t.

Test plan:
[1] Run t/db_dependent/Upload.t
[2] If you only apply this patch, you can also test uploading with a
    user that has edit_catalogue but does not have tools or circulate.
    Upload status should say: Denied.

Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
2016-04-27 16:14:16 +00:00
0c138700b2 Bug 14893: Separate temporary storage per instance in Upload.pm
To make life easier when multiple instances are uploading temporary
files to Koha, this patch adds the database name to the upload subfolder
in your /tmp folder.

Note: Although multiple instances could share the same subfolder for
temporary storage (hashvalue is based on a timestamp too), it will be
better to separate them for efficient housekeeping (removing older or
partial files with a cronjob etc.)
Since multiple instances come with separate permissions, keeping them in
separate folders will be much simpler.

Permanent storage is not affected by this patch. The location of permanent
storage is ruled by the upload_path in each config file. Sharing that
space is not recommended too.

Although it may not be strictly necessary yet to remove files from the old
temp storage folder (before the 3.22 release), the accompanying db rev
performs that housekeeping task.

Test plan:
[1] Do not yet apply this patch. Upload a temporary file (use the
    tools/upload.pl script without selecting a category).
[2] Check /tmp/koha_upload.
[3] Apply this patch. Run the db rev with web installer.
[4] Upload another temporary file.
[5] Check /tmp for folder [your_database]_upload.
[6] Check that /tmp/koha_upload is gone and the associated records too.
[7] Run the adjusted t/db../Upload.t

Followed test plan. Could not do steps before applying patch [1]-[2]
(I suppose a local permission problem).
After applying patch and updating db file appears in /tmp/[database]
as expected.
Signed-off-by: Marc Véron <veron@veron.ch>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2016-01-29 01:17:55 +00:00
9f753799d8 Bug 14321: [QA Follow-up] Additional QA requests
This patch makes following requested changes:

[1] Renamed categorycode to uploadcategorycode (with atomicupdate file)
[2] Confirmation before deleting an uploaded file
[3] Label for Public checkbox can be clicked
[4] Back button on result list (not in plugin mode)
[5] Correct additional variable interpolation from sql queries

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested the db revision, unit test.
Uploaded temporary and permanent file, incl. plugin mode.
Deleted an upload.
Ran stage-marc-import.pl.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-09-25 12:02:52 -03:00
ba62a7ef24 Bug 14321: Introduce Koha::Upload
This patch introduces Koha::Upload. It will replace the modules
C4::UploadedFile.pm and the new C4::UploadedFiles.pm (from BZ 6874).
It also includes a new unit test.

NOTE: This unit test will replace the test for UploadedFiles.pm. It will no
longer use dependency Test::CGI::Multipart. We are now mocking CGI and its
hook to achieve the same result.

Test plan:
[1] Run t/db_dependent/Upload.t.
    Note that if you see a WARNING, you will still need to add an entry
    upload_path to your koha-conf.xml. Or you need to give write
    permission to your Koha instance user for that folder.

Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-09-25 12:02:51 -03:00